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EVALUATION 


This  report  describes  the  software  development  technology  and 
management  practices  employed  on  three  specific  developments  by  Computer 


Sciences  Corporation. 

The  intent  of  the  RADC  program  to  which  this  document  relates, 

TPO  V/3.4,  is  to  describe  and  assess  software  production  and  management 
tools  and  methods  which  significantly  impact  the  timely  delivery  of 
reliable  software. 

The  study  contract  is  one  of  a series  of  six  with  different  firms 
having  the  similar  purpose  of  describing  a broad  range  of  techniques 
which  have  been  found  beneficial. 

RADC  is  engaged  in  promoting  utilization  of  Modern  Programming 
Technology,  also  called  Software  Engineering,  especially  in  large 
complex  Command  and  Control  software  development  efforts. 


UJ. 

ROG&  W.  WEBER 
Project  Engineer 


SECnON  I - RESEARCH  TASK 


1.1  OVERVIEW 

The  primary  objective  of  this  research  task  was  to  assess  the  effects  of  modern 
programming- practices  on  selected  t'omputer  Sciences  Corporation  (CSC) 

software  system  development  projects.  In  order  to  do  this  it  was  necessary  to 
identify  and  define  specific  MPP  used  on  the  projects  studied.  MPP  are  those 
engineering  and  management  techniques  that  may  be  applied  in  an  integrated  fashion 
to  influence  software  reliability,  cost,  and  schedule  during  development  of  a software 
system.  Tliis  definition  is  broad  enough  to  include  all  of  the  diverse  management 
and  technical  staff  activities  such  as  structured  programming,  design  reviews, 
support  software  implementation  and  others.  Each  project  was  carefully  researched 
to  isolate  the  programming  practices  used.  In  some  cases  the  project  managers 
had  adapted  specific  practices  to  meet  unique  requirements.  These  variants  were 
also  identified  and  defined.  Using  each  complete  list  of  project  MPP  as  a guide,  the 
individual  projects  were  carefully  examined  in  depth.  All  available  information 
sources  were  used.  'Hiis  included  project  personnel,  documentation,  reports, 
products,  etc.  This  research  was  directed  toward  assessing  the  impact  of  the  pro- 
gramming practices  on  the  conduct  and  completion  of  the  system  development.  The 
product  of  this  research  is  a complete  analysis  of  the  three  CSC  projects  which 
served  as  the  basis  for  the  subsequent  research  tasks. 

Once  the  assessment  had  been  com]iletetl,  the  programming  practices  identified  were 
compared  with  the  structured  programming  technology  (SPT)  techniques  discussed 
in  the  IBM  Structured  Programming  Series.  Before  the  two  could  be  contpared  it 
was  necessary  to  develo[)  a format  and  criteria  for  completing  the  comparison. 

Tliere  was  not  a one-to-one  correspondence  of  MPP  to  SPT  techniques  except  in  a 
few  instances.  In  some  cases,  terminology  was  different  where  actual  activities 
were  the  same  or  very  nearly  alike.  In  these  instances  the  practice  and  technique 
were  identified  by  their  similarities.  In  every  instance  [lossibic,  correspondence 
between  individual  MPP  and  SPT  techniques  was  established  for  comparison  purposes. 
The  comparison  process  was  completixi  for  each  a|i|ilicab!e  computer  program  life 
cycle  phase,  and  to  the  extent  [xissible,  was  based  upon  quantifiable  factors  common 
to  the  activities  under  study.  Where  correspondence  was  not  established  the 
activity  was  evaluated  and  compared  to  other  MPP  and  SPT  techniques  within  ajipro- 
priate  phases. 

For  the  purpose  of  this  stud\'  the  eompuier  program  life  cycle  as  describcil  in  the 
AFM  800  series  was  used.  Ibis  manual  defines  the  life  cycle  as  being  composed  of 
.Analysis,  Design,  (Txie  anil  Checkout.  Test  and  Integration,  Installation,  and 
Operating  and  Support  phases.  The  Design  phase  has  been  sulxJivided  into  Functional 
and  Detailed  Design  subphases  for  the  purpose  of  these  discussions.  'I’he  phases 
are  shown  on  I'able  1-1  along  with  phase  activities,  documents,  and  reviews. 


Table  1-1.  Computer  Program  Life  Cycle 


Following;  the  comj.'arison  of  MI’l^  and  Sl^  T tec  hniques,  a composite  set  of  standard 
practices  was  identific3d.  These  standard  practices  were  selectc3d  because  of  their 
positive  effect  on  the  acquisition  process  and  their  utility  within  the  various  types 
of  system  development  environments.  'Ifie  factors  influencing;  the  selection  of 
these  practices  are  discussed,  along  with  techniques  for  their  use.  'Ihis  set  of 
modern  programming  practices  represents  an  0[jtimum  array  of  management  and 
technical  aids  to  be  applied  throughout  the  c-omplcte  life  cycle.  However,  these 
practiees  are  not  representative  of  the  entire  spectrum  of  available  management 
and  technical  aids.  Thev  were  selected  from  only  three  projects  and  theStructured 
l^rogramming  Series.  It  must  he  emphasizetl  rhat  there  are  other  practices  which 
should  be  evaluated  in  order  to  produce  a comprehensive  analysis. 

'Fhe  final  phase  of  the  Software  Prodix;tion  Data  pnjject  involved  development  of  a 
plan  for  future  MPP  research.  Ifie  research  plan  presents  a phased  a|>proach  to 
the  study  of  IVIPP  that  is  an  extension  and  amplification  of  research  conducted  under 
this  contract.  'Hie  MPP  study  diseuss(xl  herein  protiuced  significant  insight  into 
the  identification  and  definition  of  effective  programming  practices.  There  remain, 
however,  many  areas  of  potentially  productive  research  that  can  be  pursued.  Tlie 
goal  for  this  research  will  be  to  confirm  the  validity  and  usefulness  of  the  i\IPP 
selection  meth<jdology.  To  accomplish  this  C.SC  proposes  a research  program  that: 

• Provides  for  systematic  |)erformance  data  collection  from  new  software 
acquisition  projects  of  varied  types 

• Defines  an  automated  data  reduction  system  to  assimilate  the  research 
data 

• Performs  anal  \ sis  of  the  reduced  performance  data.  > 

'fhe  .MPP  research  was  haniperecl  during  its  comluct  by  adverse  circumstances  which 
had  some  effect  on  the  results.  This  is  not  to  say  that  the  conclusions  drawn  from 
the  research  arc  nccessarile  invalid.  It  is  simply  to  point  out  that  for  conclusive 
proof  that  the  research  findings  arc  true,  more  comprehensive  and  extensive  re- 
search is  necessa'’y.  The  most  significant  problems  encountcrai  in  this  research 
project  were  twofold.  First,  none  of  the  three  projects'  management  controls  or 
reporting  protluced  the  types  of  data  necessary  for  a thorough  ana'ysis  of  the  pro- 
gramming practices  cffecti\ encss . Die  data  requirid  for  complete  analysis  must 
be  defined  at  the  project  outset  and  then  collected  and  verified  throughout  the  life 
of  the  project.  I'lie  performance  measurement  and  reporting  required  for  this  goes 
far  beyond  that  necessary  for  routine  project  managcir.ent  and  wilt,  in  all  likelihood, 
increase  overheael  costs  considerabl'  . Secemd,  bv  beginning  this  study  when  the 
projects  We’re  completed  or  in  the  final  dcwilopment  phases  muedi  ol  the  information 
sought  was  not  readily  available.  Key  project  personnel  often  hael  transferred, 
substantiating  reisource  data  was  occasionally  missing  and  some  reports  were 
diffiegilt  to  analyze. 


MPP  data  collection  can  only  be  effective  if  it  is  structured  as  a coordinated  project 
task  which  is  an  integral  part  of  the  software  development.  TTiis  approach  to  soft- 
ware data  collection  offers  significant  benefits  over  that  used  for  this  study.  First, 
by  initially  defining  data  collection  requirements  and  procedures  there  is  more 
assurance  that  they  will  be  less  disruptive  of  the  software  development  process  and 
will  produce  more  definitive  and  accurate  data.  It  is  extremely  important  that  data 
collection  not  inhibit  the  normal  production  cycle.  Data  accuracy  must  be  main- 
tained throughout  the  collection  process  to  insure  the  validity  of  research  findings. 
Next,  software  data  collection  must  be  carried  out  by  an  independent  research 
group.  To  the  maximum  extent  possible  members  of  this  group  should  maintain 
a disinterested  status  relative  to  the  project  in  order  to  prevent  any  bias  from 
appearing  in  the  research  reposrt.  'ITiis  approach  will  be  costly  in  terms  of  man- 
power required  in  excess  of  that  required  exclusively  for  program  development. 

L2  REPORT  CONTE xNT 

This  technical  report  includes  both  the  results  of  the  MPP  research  and  the  future 
research  plans.  Section  2 contains  descriptions  of  the  three  CSC  projects  studied, 
including  definitions  of  the  MPP  used  on  each  project.  Section  3 includes  the  analy- 
ses of  the  practices  used  on  these  three  projects  plus  comparisons  among  related 
MPP  from  these  projects.  Section  4 discussas  the  CSC  comparison  methodology. 
Section  5 defines  a recommended  set  of  programming  practices  selected  from  MPP 
and  SPT  techniques.  Section  6 is  the  plan  for  future  research  into  MPP  technology. 
Appendix  A contains  a list  of  terms  common  to  the  discussions  of  programming 
practices  included  in  this  report. 
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2.1  OVERVIEW 

The  three  CSC  projects  studied  had  many  similar  characteristics,  and  yet  they  were 
diverse  enough  to  have  varied  environments  for  employment  of  MPP.  Each  project 
involvai  either  direct  or  indirect  contractual  relationships  witli  the  C.S.  Nav'j'.  The 
software  in  all  three  projects  was  developefi  for  real-time  command  and  control 
systems  for  shipijorne  employment.  Tlie  earliest,  and  largest  (in  terms  of  manning) 
of  the  three  provided  the  nucleus  of  CApericncai  personnel  for  the  other  two.  Also, 
much  of  the  support  and  test  software  was  common  to  all  three.  This  was  due  to 
the  standardization  of  computer  hardware  and  software  for  Naval  tactical  systems. 

Ihe  Univac  AN/CVK  7 computers  are  used  for  Navy  shipbornc  tactical  systems  and 
all  software  is  developetl  using  the  CMS-2  compiler,  'llm  CMS-2  compiler  is  the 
responsibility  of  one  organization  that  makes  all  modifications  and  enhancements 
to  the  compiler  to  ensure  all  Navy  installations  are  operating  with  a current  standard 
version.  Nevertheless,  development  environments,  mam^ement  approaches  and 
system  reH|uircments  all  were  different,  and  the  success  achieved  in  each  of  these 
acquisition  projects  variul  considerably.  The  following  discussions  will  provide 
a basis  for  the  MPP  analysis  and  recommendations.  Table  2-1  identifies  the  three 
projects  and  the  |irogramming  practices  they  em|iloyed. 

'Hie  subsections  which  follow  include  discussions  of  .KEGLS  (2.2),  TRIDENT  (2.3) 
and  DD-063  (2. -I).  These  discussions  will  cover  recjuirements , environment, 
history  and  MPP. 

2.2  AEGIS  ENGINEERING  DEVELOPMENT  MODEL-]  PRO-IECT 

The  .AEGIS  Engineering  Development  Model-1  (EDM-1 ) projech  was  a subcontract 
with  RCA  for  the  L..S.  .Navy  to  design  and  develop  engineering  naxiels  oi  the  Command 
and  Control  (C  and  C)  subsystem,  Radar  (SPY-1)  subsystem.  Operational  Readiness 
Test  System  (OR '!>>),  the  real-time  executive  (.A  TEP)  and  the  suijjmrt  software 
(i.e..  Signal  Processor  Checkout  (SIPCO),  Ltility  software  (.A'll  l.i  and  data  re- 
duction programs  (SM.AIf  1'  and  RAP)). 

2.2.1  System  Requirements 

AEGIS  is  an  integrated,  computer-controlled  shij-jboard  weapon  system  composed 
of  a guided  missile,  acquisition  and  gaiidance  radar  and  an  automated  control  system. 
EDM-1  is  a proof-of-concept  system  now  at  sea  on  the  CSS  Norton  Sound.  'Ihe 
computer  sel octal  for  operational  software  was  the  END' .At.'  .AN  TYK-7  and  the 
language  selected  was  the  CMS-2  high  oiTlcr  language. 
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Table  2-1.  Project  Programming  Practices 


TOP  DOWN  DESIGN 


THREADS 


BUILDS 


THREADS  MANAGEMENT  SYSTEM 


PROJECT  REVIEWS 


X 


X 


X 


PROGRAMMING  TECHNIQUES 


PROGRESSrV'E  TESTING  & INTEGRATION 


SOFTWARE  CONFIGURATION  MANAGEMENT 


STOUCTURED  WALKTHROUGH 


BUH.D  LEADER 


CHIEF  PROGRAMMER 


SUPPORT  PROGRAMS 


VERIFICATION  PROCEDURES 


INDEPENDENT  TEST  & EVALUATION 


X 


X 


X 


X 


X 


X 
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'iTie  CSC  contract  [jerCormance  period  evaUiatcxi  l)ytlii.s  research  project  was  Irom 
January,  1970  , through  April,  197.'),  for  the  KD.M-1  version  of  the  system.  'Ilie 
data  colleetetl  for  this  research  only  covers  the  K1)M-1  software  development, 
howov'er,  CSC  also  provided  lev('l  of  idfort  (l  Adl)  supjiort  to  KCA  in  the  areas  of 
system  integration  and  testing  ot  1J)M-1  soltware. 

'file  AEGIS  technical  staff  average  approximately  one  hundred  personnel  over  the 
contract  jicriod.  lliis  staff  had  an  average  of  ten  years  data  processing  experience 
and  90'/,  had  related  exi)erience  prior  to  assignment  on  the  AEGIS  project.  Related 
c.xpcrience  was  determined  to  he  experience  in  the  subsystem  to  which  the  person 
was  assigned  (i.e.,  n al -time  executive,  radar,  command  and  control,  etc.).  The 
staff  was  comprisid  of  07','  senior  members  of  the  technical  staff  or  higher,  while 
70%  of  the  staff  possessed  college  degrees.  Ihe  requirements  tor  a senior  member 
of  the  CSC  technical  staff  are;  RS  degr  ee,  MS  prefernd.  Minimum  five  years 
professional  experience  with  suiicrvisory  or  management  e.xpcrience.  Ihe  average 
turnover  rate  for  the  technical  staff  during  contract  periotl  was  21 . per  year. 


2.2.2  I Toject  History 

Ihe  EDM-1  was  an  R and  1)  effort  with  customer  emphasis  on  quality.  The  hardware 
and  software  for  the  system  were  developed  in  irarallel  which  resulted  in  several  | 

restarts  and  modifications  to  the  original  design.  Ihe  project  personnel  interviewed 
during  this  research  felt  that  the  success  of  this  project  was  due  to  the  ver\  close 
working  relationship  among  all  personnel  and  a good  overall  system  cooidination. 

The  CSC  developtd  software  amountal  to  over  .210,000  words  of  core  in  size  at  a 
cost  of  S7,Sr, 0,000  for  the  development. 

'Ihe  first  year  of  the  contract,  1970,  was  spent  in  preliminary  and  critical  design 
reviews.  'Ihe  executive  design  was  accepted  in  iUarch,  1!)71,  the  command  and 
control  design  was  accepted  in  May,  1971,  and  the  radar  (Sl'Y-D  design  was 
accepited  in  September,  1971.  'J'lu;  build  concept  \eas  usvd  on  the  AEGIS  project  to 
permit  demonstnition  of  system  cai)abilUii’S  to  tiie  customer  a.s  the  capabilities  were 
producal.  'Ihe  first  capability  of  the  command  and  control  (C  and  C)  system  was 
demonstrated  in  Xo\  ember,  1971.  The  lirst  SP\-1  radar  capabilities  wei  e 
demonstrated  in  January,  1972,  and  C and  ('  :intl  Slh-1  interlace  c;apabilities  were 
successfully  completfjd  in  Eebruar\  , 1972.  'Ihe  fin. J prcduclion  '.ersionwas  tested 
and  subsystem  integration  completed  in  earl\  1972  ,,nd  del i\'i>rof.I  to  the  customer 
in  April,  1972. 


'Ihe  major  milestones  were  met  earl>  or  on  time.  Ihc  develo|)mcnt  schalui  e was 
established  by  CSC  in  conjunction  with  the  customer  and  monitored  very  closely. 

'Ihe  demonstrations  of  •''i  rtial  system  capability  (l)uilds)  were  the  intermediate 
milestones  and  these  w.„re  nu  t on  time  or  within  a week  oi-  two  of  the  scluxiuled  date. 


The  CSC  management  monitored  this  progress  very  closely  and  redistributed 
resources  when  necessary  to  maintain  the  overall  schedule. 

2.2.4  Project  MPP 


The  modern  programming  practices  identified  on  the  AEGIS  project  were: 

• Top  Down  Design 

• ITi  reads 

• Builds 

• Threads  Management  System 

• Project  Reviews 

• Functional  Programming 

• Progressive  Testing  and  Integration 

• Software  Configuration  Management 

• Support  Programs 

• Independent  Test  and  Evaluation 

2. 2. 4. 1 Top  Down  Design 

The  AEGIS  designers  decided  on  a top  down  approach  to  the  software  system  design 
for  several  reasons,  the  most  important  being  the  system  size  and  complexity.  The 
AEGIS  system  was  at  that  time  one  of  the  largest  ever  developed  by  CSC  and  the 
top  down  design  was  to  provide  a systematic  approach  to  segment  the  system  into 
manageable  development  units.  The  decomposition  and  refinement  of  functions  down 
to  the  program  level  during  a top  down  design  requires  the  identification  of  the 
interfaces  necessary  to  communicate  between  the  levels  of  the  system.  Once  these 
interfaces  between  levels  of  the  system  are  identified  the  system  becomes  less 
complex  as  components  become  smaller.  In  contrast,  the  bottom  up  approach 
becomes  more  complex  as  the  next  higher  level  of  a system  is  built  and  lower  levels 
dictate  what  system  interfaces  are  required.  Thus  Top  Down  Design  was  chosen 
for  AEGIS. 

The  total  system  development  concept  for  the  AEGIS  program  was  to  develop  the 
EDM-1  version  for  maximum  functional  flexibility  and  system  maintainability  to 
enable  future  enhancements  to  be  made  with  a minimum  effort  and  cost.  The  top 
down  design  provided  CSC  with  a system  architecture  that  was  compatible  with 
development  of  functional  subsystem  components. 

2. 2. 4. 2 THREADS 

The  THREADS  concept,  which  is  used  by  CSC  on  most  of  its  current  projects,  was 
conceived  during  the  AEGIS  project.  AEGIS  was  one  of  the  largest  real-time  systems 
CSC  had  ever  developed  and  a technique  was  needed  to  ensure  that  all  system 
requirements  were  covered  bv  the  design  specification.  Some  of  the  AEGIS  staff 

2-4 


members  developed  this  technique  by  identitvin.i;  all  system  inputs  and  outputs  in 
the  desi>^n  document  and  identityinj^  the  processes  or  paths  through  the  system  from 
input(s)  to  outi->ut(s).  'Hien,  these  processes  were  mapped  back  to  the  rujuirements 
to  verify  that  requirements  were  met  by  the  design,  thus  insuring  that  the  design 
was  complete. 

Once  the  data  from  tliis  first  threading  process  had  been  c'o!lect«i  and  documented 
the  true  potential  of  threads  became  apparent.  The  grouping  of  ri^lated  processes 
or  threads  was  used  to  identify  functional  system  components  that  coubj  be 
developed  and  demonstratul  as  a i)artial  system  capal/il it',  ; tliese  groiips  were 
callal  Builds.  Since  the  threads  were  paths  Ihrou.gh  the  system,  the  interfaces  be- 
tween modules  had  to  be  identified  and  tlie  data  base  elements  used  for  intermodule 
communications  were  defined  early  in  the  development  cycle.  Hie  early  identifi- 
cation and  definition  of  mcKlule  interfaces  establislud  the  message  |)atterns  for  all 
programs  which  eliminattd  i)rogrammer  interiu'etation,  thus  reducing  this  type  of 
integration  problem. 


2. 2. 4.. 3 Builds 

The  Build  concept  was  primarily  used  on  the  AllGlS  project  as  a means  of  incre- 
mental system  development.  .A  build,  as  statoil  in  section  2. 2. 4. 2.  w as  a group  of 
functionally  related  threads  which  could  bedevelopa!  and  demonstrated  as  a partial 
system  capability.  By  grouping  tlic  threads  into  controllable  entities,  the  total 
software  production  could  be  schululod  and  controlled  at  a manageable  level.  This 
technique  allowed  management  to  schedule  the  development  basoLi  on  hardware 
availability,  critical  areas  of  software  and  other  constraints  and  to  util ize  develop- 
ment resources  to  the  maximum  degree,  'i'his  technic|ue  provide*.!  visibility  if  re- 
planning was  necessary  due  to  unavoidalile  problems,  such  as  late  delivery  of  a 
hardware  component.  4’he  build  completion  dates  wtu'e  used  as  intermediate  mile- 
stones by  lower  levels  of  management  to  measure  prc>gress. 

The  technique  also  promoted  progressive  testing  and  integration  which  is  covered 
in  section  2.2.4. 7 of  this  report. 

2. 2. 4. 4 'Ilireads  Management  System  fTMS) 

As  discussed  in  Section  2. 2. 4. 2 of  this  report,  after  the  threads  were  defined  and 
documented,  other  uses  of  this  information  were  conci;i\al  and  computer  |)i’ograms 
were  devclopixi  to  obtain  the  full  benefits  of  the  data,  lliis  collection  of  programs 
was  Called  the  llu’eads  Management  .System  ('i'MS')  because  man\  of  the  reports 
generated  by  the  system  were  used  for  management  control. 


'Hie  data  base  for  the  'I'MS  contained  identifying  information  about  each  system  thread, 
such  as; 


• Thread  name 

• lliread  number 

• Performance  specification  paragraph  reference  number 

• Design  specification  paragraph  reference  number 

• Modules  required  to  process  the  thread 

• 'lliread  input  source 

• 'Thread  output  format 

• 'Thread  status  such  as  designed,  coded,  tested  and  integrated 

• Milestone  reporting  dates 

'Ilie  AEGIS  management  used  the  TMS  report  to  monitor  production  progress  of 
modules  by  threads  within  a module.  The  status  is  reported  in  a binary  format, 
a module  is  either  100%  coded  or  0%  coded.  'This  type  of  status  reporting  eliminates 
estimates  by  the  programmers  and  reports  the  true  status  of  a software  package, 
y,  - 'The  system  also  reports  missed  schedules  on  an  exception  basis,  as  well  as,  current 

» and  projected  activity  status.  Other  reports  include  any  changes  to  the  data  base, 

_ such  as  rescheduling  of  milestones  and  providing  change  impact  analysis  for  pro- 

‘ posed  system  changes. 

'Iliis  system  has  been  enhanced  since  its  development  on  the  AEGIS  project  to 
provide  extensive  management  reports.  'Hie  system  served  AEGIS  as  a control 
tool  and  provided  visibility  to  CSC  management  and  the  customer. 

2. 2.4. 5 Project  Reviews 

Project  reviews  may  not  always  be  considered  to  be  MPP,  however,  during  this 
research  all  project  managers  interviewed  felt  that  the  use  of  in  process  reviews 
(IPR)  kept  the  customer  involved  in  the  project  throughout  the  development  cycle 
and  enhanced  development  progress.  The  normal  sequence  of  events  for  most 
projects  begins  with  the  customer  approving  the  system  design  at  the  Critica’ 

Design  Review  (CDR)  and  then  departing  until  some  time  later  when  he  returns  to 
accept  the  software.  'Tliis  results  in  little  or  no  input  by  the  customer  as  the  pro- 
duct evolves  through  the  development  phases.  'Hiis  sometimes  causes  system 
integration  problems  due  to  hai’dware  or  software  changes  or  trade-offs  being  made 
without  proper  developer-customer  coordination.  CSC  encourages  customer 
involvement  during  all  phases  of  the  software  development.  'This  fosters  awareness 
by  all  system  development  participants  in  problem  areas  thus  promoting  their 
expedient  resolution  and  keeps  everyone  informed  of  the  current  status  of  progress 
on  all  project  tasks,  llie  AEGIS  project  staff  had  a monthly  status  meeting  in  which 
problem  areas  were  identifiod,  action  items  were  assigned  to  the  appropriate 
personnel  and  progress  reported  since  the  last  meeting.  CSC  staff  personnel  en- 
joyed a very  good  working  relationship  with  the  other  members  of  the  AEGIS  program 
team.  'This  is  not  to  imply  there  were  no  problems,  however,  a good  overall  team 
coordination  assisted  in  quick  solutions  or  work  arounds  to  most  problems. 


w 
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The  preliminary  design  reviews  (PDlls)  and  critical  design  reviews  (CDRs)  are 
contract  obligations,  however,  the  format  in  which  they  are  performed  can  have  a 
significant  impact  on  the  system  development.  'ITie  schedule  for  FDRs  and  CDRs 
are  normally  milestones  in  a development  cycle  (and  should  be),  however,  the 
schedule  should  be  flexible  enough  to  be  adjusted  if  the  design  is  not  100']?  complete. 
To  hold  a review  on  a scheduled  date  without  regard  to  actual  design  status,  often 
results  in  reviewing  an  incomplete  document  which  creates  excessive  questions  and 
wastes  many  hours  of  valuable  management  time.  If  approval  is  given  to  an  incom- 
plete design  document  the  result  will  probably  be  an  incompl  ete  or  faulty  system 
design. 

'Hie  CSC  approach  to  this  problem  is  to  insure  the  design  is  complete  prior  to  the 
review  or  delay  holding  the  review  until  it  is  complete.  Copies  of  the  material 
to  be  reviewed  are  sent  to  all  parties  attending  the  review  early  enough  to  allow 
them  time  to  read  the  material  and  formulate  questions  prior  to  the  meeting.  This 
procedure  expedites  the  review  process  and  produces  a solidified  design. 

2.2.4.  (j  Functional  Programming 

'Fhe  AFGIS  program  was  designed  as  a three  phase  procurement  to  ije  developed  in 
engineering  development  models  by  adding  enhancements  and  additional  capabilities 
to  each  succeeding  model.  Therefore,  CSC  designed  the  software  in  functional 
modules  to  enabl'^  additional  functions  to  be  added  or  upgrade  existing  functions 
without  affecting  the  other  system  components.  The  system  design  technique  was 
top  down,  however,  the  module  development  was  not  restricted  to  hierarchical 
levels  of  the  system. 

By  programming  modules  to  perform  a function  or  functions,  conversion  to  the 
next  phase  or  model  in  the  AEGIS  program  required  fewer  development  resources. 
'Ibis  technique  also  allowed  flexibility  during  development  of  EDM-1  since  require- 
ments changed  rapidly  as  they  normally  do  in  an  R and  D environment.  Another 
consideration  with  this  approach  was  that  it  supported  progressive  testing  and  inte- 
gration which  is  discussed  in  Section  2. 2. 4. 7. 

Structured  programming  (SP)  was  considered  for  this  effort,  but  was  not  used. 
Several  test  programs  were  written  using  the  SP  constructs  and  the  same  programs 
were  written  in  free  form.  The  results  of  these  tests  revealed  that  SP  written  in 
CMS-2  (the  high  order  language  selected  for  tWs  project)  required  more  core 
than  free  form.  Since  the  core  allocation  for  the  system  was  limited,  the  decision 
was  made  not  to  use  SP.  CSC  did  enforce  coding  conventions  on  fhe  tactica'  sub- 
system programs  to  ensure  standardized  module  construction  and  to  promote  main- 
tainability of  these  programs,  while  obtaining  the  most  effective  use  of  core. 
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2.2.4.  7 Progressive  Testing  and  Integration 


* 


The  CSC  AEGIS  staff  used  the  progressive  testing  and  integration  technique  in 
conjunction  with  the  MPP  discussed  earlier  to  support  phased  system  implementation. 
Progressive  testing  and  integration  is  the  process  of  testing  the  builds  as  they  are 
developed,  demonstrating  the  capability  performed  in  the  build  and  baselining  the 
;s  programs  after  a successful  demonstration.  This  process  continues  as  each  suc- 

cessive  build  is  completed,  with  each  build  being  integrated  with  the  previously 
’ baselined  builds  and  additional  system  capabilities  being  added  until  the  total  system 

^ is  integrated. 


The  benefits  of  this  tool  make  it  very  cost  effective.  For  example,  errors  are 
detected  and  corrected  earlier  in  the  development  cycle  when  the  cost-to-correct  is 
lower.  By  initiating  the  integration  process  early,  any  interface  problems  are 
easier  to  track  and  total  system  integration  at  one  time  is  avoided. 

Another  benefit  is  that  the  customer  can  see  success  demonstrated  early.  During 
the  AEGIS  demonstrations,  Navy  personnel  were  used  to  operate  the  consoles  to 
demonstrate  that  the  software  and  the  man-to-machine  interfaces  functioned  as 
designed.  This  technique  gave  the  Navy  hands-on  experience  and  their  inputs  and 
comments  helped  to  improve  areas  such  as  display  formats  and  operator  key-ins. 
TTie  build -a -little,  test-a-little  philosophy  of  incremental  development  and  testing, 
used  in  conjunction  with  regressive  testing  to  assure  that  previously  baselined 
functions  were  not  affected,  provides  a complete  verification  of  the  system  functions 
prior  to  system  testing  and  acceptance  tests, 

2.2.4. 8 Software  Configuration  Management  (SCM) 

As  stated  earlier  in  this  report  AEGIS  was  a very  large  system  and  contro'  of  the 
software  during  the  phased  implementation  process  required  a large  SCM  staff. 

The  AEGIS  SCM  staff  ranged  between  four  and  six  personnel  over  the  life  of  the 
project.  The  primary  i esponsibility  of  the  SCM  staff  was  to  control  the  baselined 
software  from  unauthorized  modifications  and  provide  documented  verification  of 
the  various  versions  of  the  system.  'Phis  group  also  tracked  and  controlled  all 
changes  to  the  computer  programs  which  had  been  baselined  and  all  system 
documentation. 

TTie  benefits  of  this  function  are  normally  dependent  on  the  size  of  the  SCM  staff 
in  relation  to  the  size  of  the  project.  'The  AEGIS  project  had  proper  funding  to 
allow  for  an  SCM  staff  capable  of  controlling  the  software. 

2. 2.4. 9 Support  Programs 

AEGIS  support  programs  were  developed  to  influence  the  design  development,  test, 
integration  and  validation  of  the  operational  software.  ITiev  w'ere  not  a part  of  the 
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operational  software.  Included  were  those  programs  used  in  the  integration  of 
hardware  and  computer  programs,  the  installation  and  checkout  of  each  system 
with  every  other  system  element.  These  programs  along  with  the  tactical  executive 
program  provide  facilities  and  tools  to  determine  product  quality  and  acceptability. 
'Hie  programs  used  included  the  following; 

• Test  Support  Programs 

• Interface  Simulators 

• Operational  Readiness  Test  System 

2.2.4.  9.  1 Test  Support  Programs 

lire  computer  program  development  activity  utilized  test  support  programs  as  tools 
in  the  process  of  development  of  computer  programs.  One  such  test  support  pro- 
gram was  the  Software  Test  and  Evaluation  Program  (STEP).  S'FEP  included  the 
following  functions; 

• Driver-Control  execution  of  STEP  internal  functions,  together  with 
module  under  test. 

• Tnterpreter-Interpret  input  control  card  images  and  set  up  schedule 
table  linking  modules  with  each  other  and  with  their  test  data. 

• Loader-Link-Editor  - Load  and  patch  the  modules  being  tested  and 
link  them  to  any  previously  loaded  modules.  ITie  loader  shall  maintain 
a memory  map  and  make  available  the  locations  of  globally  defined 
symbols. 

• I/O  Control  - Control  the  input-output  section  of  the  computer. 

• Test  Evaluation  - Evaluate  and  print  specified  test  results. 

• SNAP/DUMP  - List  contents  of  selected  areas  of  memory. 

Other  programs  like  STEP  were  developed,  as  needed,  to  serve  as  test  tools  to  aid 
in  the  orderly  development  of  computer  programs  for  the  AEGIS  system. 

2.2.4. 9.  2 Interface  Simulators 

Computer  program  debugging  and  testing,  as  well  as  system  debugging  and  testing, 
were  aided  by  use  of  interface  simulation  programs  in  the  absence  of  complete 
system  interfaces.  'ITiese  programs  introduced  external  stimuli  into  the  system, 
program  or  element  under  test,  noting  occurrance  and  validity  of  response;  facili- 
tated action/reaction  criteria;  and  supported  evaluation  of  timing  considerations. 
'Hie  utility  portion  of  STEP  was  used  as  a test  tool  along  with  simulation  programs, 
as  appropriate. 

The  complement  of  interface  simulator  routines  includetl  both  special-purpose 
routines  for  directly  simulating  the  interfaces  to  the  routines  being  tested  and  the 


a 


1 


r< 

f 

I 

» 

I* 

i. 

I 

! 

i 

! 


V, 


2-9 


service  routines  that  were  used  to  help  set  up  and  monitor  the  particular  test.  The 
former  set  was  prepared  individually  for  each  autonomous  system.  Some  of  the 
service  routines  were  shared  by  more  than  one  simulator  package, 

2,2.4. 9.3  Operational  Readiness  Test  System  (ORTS) 

Although  ORTS  was  configured  as  part  of  the  operational  software  it  was  classified 
as  a support  program. 

ORTS  provided  a system  for  performance  and  readiness  monitoring,  fault  diagnosis 
and  maintenance  evaluation  of  function  availability  and  operability  level,  and 
summary  status  reporting  at  the  command  level.  Within  this  framework,  the 
operational  programs  residing  in  the  computers  employed  by  the  radar,  command 
and  control  and  weapon  direction  subsystems  demonstrated  the  following  capabilities 

• Fault  detection  by  periodic  monitoring  and  analysis  of  critical  sensor 
test  data  and  computer  programs. 

• Fault  isolation  by  monitoring  and  analysis  of  discrete  sensor  test  data 
and  computer  programs, 

• System  operability  testing  by  processing  and  analysis  of  simulated  radar 
return  data. 

• Controlled  injection  of  simulated  signals  into  monitored  equipment  for 
the  purposes  of  fault  detection,  fault  isolation,  and  operability  testing. 

• Control  of  ORTS  displays  for  the  purpose  of  reporting  the  results  of 
testing  and  processing  of  ORTS  operator  inputs,  for  test  initiation,  and 
access  of  system  operability  and  maintenance  action  information. 

• System  operability  testing  of  the  AEGIS  equipment. 

• System  operability  testing  of  the  overall  AEGIS  system. 

2.2.4.10  Independent  Test  and  Evaluation 

Computer  program  testing  was  conducted  by  the  AEGIS  Test  and  Evaluation  (T&E) 
Department,  This  department  operated  independently  within  the  project  organization 
to  prepare  test  plans  and  procalures  and  conduct  the  testing.  Project  staff  pro- 
grammers and  analysts  completed  all  computer  program  unit  testing  during 
development  of  the  programs.  When  unit  testing  was  successfully  completed  the 
programs  were  integrated  into  functional  subsystems  known  as  builds  and  released 
to  the  T&E  Department  for  testing.  Using  prepared  scripts  and  scenarios  the  T&E 
personnel  evaluated  the  performance  of  build  programs  using  a build  demonstration 
as  the  vehicle  for  the  testing.  A build  demonstration  is  a formal  test  of  specific 
system  functional  capabilities.  Normally,  the  customer  observed  each  build 
demonstration  as  the  vehicle  for  the  testing.  A build  demonstration  is  a formal  test 
of  specific  system  functional  capabilities.  Normally,  the  customer  observed  each 
build  demonstration  and  reviewed  test  documentation. 
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'Fhe  T&E  Department  manager  reported  directly  to  the  AEGIS  project  manager. 
Computer  program  testing  was  executed  and  documented  under  project  guidelines 
using  standardized  procedures. 

2.3  TOIDENT  MOiMITORING/DATA  ACQUISI^nON  Sl'BSYSTEMS  PROJECT 

The  TRIDENT  Monitoring/Data  Acquisition  subsystems  project  was  a subcontract 
with  IBM  for  the  U.S.  Navy  to  develop  the  Monitoring  subsystem  (MS)  and  the  Data 
Acquisition  pA)  subsystem  software  for  the  TTIIDENT  submarine  sensor  system. 

CSC  had  res|X)nsibility  for  design  specification,  detail  design,  program  generation, 
coding  and  checkout,  and  subprogram  integration.  However,  all  system  testing 
and  integration  is  being  accomplished  at  this  time  by  CSC  personnel  at  the  Navy 
Land  Based  Evaluation  E’acility  pBE'F)  in  Rhode  Island.  The  work  is  being  done 
under  a time  and  materials  contract  with  IBM  as  the  prime  contractor. 

2.3.1  System  Requirements 

The  THIDENT  class  submarines  will  be  the  largest  of  the  Fleet  Ballistic  Missile 
Submarines  (SSBN),  and  will  carry  a complement  of  24  TRIDENT  I missiles.  Each 
submarine  is  designed  for  quiet  operation,  high  systems  reliability,  and  extended 
cruise  endurance. 

Tlie  MS  and  DA  subsystems  operate  within  the  ship’s  central  computer  complex  to 
provide  automated  sensor  signal  analysis,  supporting  quiet  operation  of  the  ship. 

The  on  lx>ard  computers  to  support  these  programs  are  the  UNIVAC  .AN/UYK-7 
and  AN/UYK-20.  The  computer  programs  are  written  in  the  CMS-2Y  high  order 
language. 

2.3.2  Development  Environment 

The  CSC  contract  performance  period  for  this  effort  was  .\pril  1974  through 
August  197C.  The  technical  staff  consisted  of  approximately  thirty  personnel  during 
the  coding  and  testing  phase,  of  these  GO'r  were  senior  level  personnel.  The  average 
data  processing  experience  for  the  staff  was  9.G  years,  and  73*^  of  the  staff 
j)ossessed  college  degrees.  Some  of  the  technical  staff  members  were  transferred 
from  sim.ilar  CSC  projects,  and  of  the  total  staff,  47C  had  directly  related  experience 
using  the  UNI\'.AC  .•^N/UYK-7  and  the  CMS-2Y  language  prior  to  assignment  to  the 
TItIDEN  r project.  The  annual  turnover  rate  was  24'J  during  the  contract  period. 

Tlie  MS  and  DA  subsystems  developed  by  CSC  totaled  94K  words  in  size  and  the 
development  cost  was  S2.4M  over  the  contract  period. 
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2.3.3  Project  Histor 


This  project  encountered  several  problems  during  the  development  cycle.  The 
major  problem  arose  when  a complete  redesign  of  the  subsystems  was  required 
after  the  sensor  data  collection  hardware  could  not  meet  the  performance  require- 
ments. The  original  hardware  known  as  the  Information  Transfer  System  (ITS)  was 
replaced  by  special  purpose  hardware  using  an  AN/UYK-20  computer  as  a control 
unit.  This  involved  a major  change  in  the  concept  of  ojjeration  of  MS  and  necessi- 
tated development  of  the  DA  subsystem.  The  decision  to  make  this  change  was 
made  only  after  the  first  two  levels  of  the  monitoring  subsystem  were  delivered 
to  the  customer. 

The  project  also  had  a problem  with  program  sizing  caused  in  part  by  the  added 
requirements  that  accompanied  the  replacement  of  ITS  by  DA  hardware.  Also  the 
subsystems  were  coded  using  structured  programming  with  the  CMS-2Y  language 
which  further  extended  the  core  budget.  Even  after  several  test  cases  proved 
that  structured  programs  required  5%  to  8%  more  core  than  non-structured  programs, 
the  decision  was  made  by  the  customer  to  continue  with  structured  programming. 

A major  effort  was  undertaken  to  reduce  the  size  of  the  programs  late  in  the 
development  cycle  and  monitoring  of  the  core  budget  became  a high  priority  effort. 

Hardware,  compiler  and  facility  problems  further  slowed  progress  and  resulted  in 
milestones  being  rescheduled.  The  customer  became  concerned  and  was  given 
additional  visibility  into  development  activities.  Additional  controls,  resources  and 
techniques  were  applied  to  the  project  and  most  of  the  problems  were  resolved. 

'Hie  problems  resulted  in  the  final  product  delivery  date  being  rescheduled  from  i 

May  1976  to  August  1976.  ^ ! 

This  project  did  not  use  the  Threads  Management  System  and  the  milestone  achieve- 
ment data  was  not  well  documented.  The  first  two  major  milestones.  Level  I and 
Level  II  of  the  original  subsystem  design  were  delivered  on  schedule.  At  this  i 

point  the  decision  was  made  by  the  customer  for  a redesign  of  the  monitoring  and 
information  transfer  subsystems  and  all  milestones  had  to  be  rescheduled. 

2.3.4  Project  MPP 

The  modern  programming  practices  identified  on  the  TRIDENT  project  were: 

• Top  Down  Design 

• Builds 

• Structured  Programming 

• Structured  Walkthrough 

• Ih'ogressive  Testing 

• Software  Configuration  Management 
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'Hie  use  of  these  MPP's  and  the  impact  they  had  on  the  CSC  TOIDENT  project  will 
be  discussed  in  this  subsection. 

2 . 3 . 4 . 1 Top  Down  Design 

CSC  was  required  by  contract  to  develop  the  monitoring  and  data  acquisition  subsys- 
tems using  top  down  structured  programming  techniques.  The  design  of  the  sub- 
systems was  implemented  using  a top  down  approach,  by  designing  the  control 
arcliitecture  first,  then  the  internal  control  logic  within  each  module  to  a progres- 
sively lower  level  of  functional  detail,  until  the  lowest  functional  level  is  complete, 

K llie  result  of  this  technique  was  a design  that  segmented  the  software  package  into 

functionally  unique  elements  that  supported  phased  development  and  implementation 
' using  the  top  down  structured  programming  techniques. 

» There  was  no  data  to  indicate  that  this  approach  was  more  or  less  expensive  than 

another  a()proach,  such  as  a bottom  up  design. 

2. 3. 4. 2 Builds 

Late  in  the  development  cyele  several  problems  began  to  surface  and  the  customer 
raiuiral  additional  management  visibility.  A CSC  technique  known  as  "builds"  was 
implemented,  'ibis  technique  groups  related  processes  into  system  functions  which 
1 can  be  demonstrated  to  the  customer.  Ibis  confirms  that  specified  fully  developed 

processes  do  perform  their  design  function  or  functions. 

^ 'fbe  technique  impacted  the  project  in  several  ways.  It  provided  the  customer  and 

CSC  management  with  visibility  into  the  true  status  of  the  project.  Ibe  milestones 
were  then  reschedulai  and  lower  level  mile.stones  were  established  to  assist  CSC 
management  in  tracking  and  controlling  the  system,  Ibis  "build-a-litti e-test-a- 
littlc"  concept  promoted  early  subsystem  integration  and  led  to  detection  of  inter- 
face problems  early.  'Ibis  technique  in  conjunction  with  other  tools  assisted  CSC 
management  in  resolving  some  problems  and  turning  out  good  software  within  a 
reasonable  time  irame. 

2.3. 4.3  Structured  Programming  (SP) 

'Ibe  use  of  structured  programming  with  the  I'NrV'AC  computers  and  CIMS-2Y  com- 
piler was  a contract  requirement.  'Ibis  requirement  created  a problem,  in  that  the 
structural  code  using  CMS-2Y  used  to  ST  more  core  than  free  form  code. 
However,  the  customer  felt  the  benefits  derived  from  structured  programming 
justified  the  additional  core  required. 
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The  CSC  analysts  interviewed  on  the  TOIDENT  project  felt  that  program  design  for 
structured  programming  was  time  consuming  because  of  the  lower  level  of  design 
detail  required.  But  the  coding  phase  was  reduced  to  almost  half  the  time  of  free 
form  programs  due  to  the  design  being  developed  at  almost  the  code  level.  The 
personnel  using  SP  for  the  first  time  thought  the  constraints  of  SP  did  not  effect  the 
productivity  of  the  senior  level  personnel  to  any  degree.  However,  a noticeab'e 
adjustment  was  required  by  the  junior  level  personnel  to  develop  logic  in  a struc- 
tured manner.  It  was  also  felt  that  better  quality  programs  were  produced  by  the 
junior  staff  members  using  SP. 

The  total  effect  of  SP  on  the  TOLDENT  programs  could  not  be  determined.  The 
measi  res  of  its  effectiveness  are  in  program  maintainability,  reduction  of  the 
testing  and  integration  effort  and  ease  of  transferring  program  responsibility. 

These  measures  could  not  be  evaluated  on  TRIDENT  since  CSC  was  not  tasked  under 
this  contract  to  perform  system  testing  and  integration  and  the  product  has  not 
reached  the  operational  phase  to  determine  maintainability. 

2.3.4.  4 Structured  Walkthroughs 

This  technique  was  used  on  the  TTUDENT  project  for  detailed  program  design  review 
and  code  review.  The  detailed  program  design  review  was  he'd  after  program  design 
and  prior  to  coding  and  the  code  reviews  were  held  after  a clean  compilation  was  ob- 
tained and  prior  to  unit  testing.  Structured  walkthroughs  were  conducted  by  an 
evaluation  committee.  'Ihe  evaluation  committee  varied  in  size  from  case  to  case 
but  normally  consisted  of  the  analysts  whose  programs  had  interactions  with  the 
program  under  review  ard  the  primary  jirogrammer's  immediate  supervisor.  The 
design  or  code  was  reproduced  and  distributed  to  the  committee  members  for  re- 
view prior  to  the  evaluation  date.  Tbe  reviews  were  internal  to  the  project  and  in- 
formal to  a degree,  to  encourage  constructive  criticism  from  the  participants.  The 
dates  set  for  the  walkthroughs  arc  part  of  the  milestones  for  completion  of  program 
design  and  coding  as  scheduled  by  the  programming  manager.  The  evaluation  team 
would  look  for  misinterpretations  of  the  data  base  structure,  communication  links, 
logic  paths  and  the  general  approach  to  the  solution  of  the  problem. 

The  effect  of  the  walkthroughs  as  expressed  by  the  personnel  on  the  staff  showed 
them  to  be  cost  effective  in  both  man-hours  and  machine  time.  The  number  of 
personnel  involved  in  a walkthough  ranged  from  three  to  seven  and  the  time  re- 
quired, was  about  four  hours  per  program.  The  direct  benefit  from  these  evalua- 
tions was  in  the  test  time  being  reduced  by  50%  to  80%  because  of  errors  which  were 
detected  prior  to  the  test  phase.  The  indirect  benefits  were  (a)  verifying  program 
interfaces  to  alleviate  integration  problems,  (1’)  spreading  knowledge  of  system 
components  among  the  staff,  (c)  accelerating  the  software  production  by  reduction 
of  test  and  integration  time,  and  (dj  detecting  errors  early  in  development  cycle, 
thus  reducing  the  cost  of  correction. 
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2.;J.4.5  Progressive  Testing 

ITiis  technique  was  used  in  conjunction  with  the  "build"  concept  for  incremental  sys- 
tem development.  'Ihere  are  several  advantages  gained  by  using  this  tool  rather  than 
developing  the  entire  subsystems,  then  testing  and  integrating  all  the  functional  com- 
ponents at  the  end  of  the  development.  This  tool  provides  intermediate  milestones 
for  tracking  progress.  It  also  allows  early  visibility  for  the  customer,  as  a build  or 
capability  is  completed  it  can  be  tested  and  integrated  with  previously  completed 
builds  to  test  interfaces  between  segments  and  demonstrate  multiple  capabilities.  If 
errors  are  detected  during  the  build  test,  they  will  be  easier  to  find  than  when  the 
entire  subsystem  is  integrated.  Likewise  interface  errors  are  easier  to  trace  during 
a phased  integration  process  since  the  number  of  interfaces  being  added  are  normally 
fewer. 

2. 3. 4.(5  Software  Configuration  Management  (SCM) 

'this  development  tool  is  not  a new  technique  in  the  software  development  process. 

Ibe  impact  it  has  on  a given  project  depends  on  the  size  of  the  SCM  staff  in  relation 
to  the  size  of  the  program  and  the  amount  of  control  the  SCM  staff  can  apply  over  the 
software  development.  The  TRIDENT  project  had  20  to  30  technical  personnel  and 
one  person  was  responsible  for  SCM  as  a part  of  his  duties.  The  prime  contractor 
did  not  specify  any  configuration  control  requirements  during  development  of  the 
software  subsystems,  therefore,  SCM  was  limited  to  recording  current  computer 
program  positions  for  testing  purposes.  The  subsystems  were  baselined  when  they 
were  delivered . 

2.4  DD-963  PROJECT 

'The  U.S.  Nav>-  is  constructing  a new  class  of  destroyers  to  assure  protection  of  the 
fleet  through  the  1980s.  This  class,  known  as  DD-963  will  be  equipped  with  the  most 
sophisticated  weapons  and  control  systems  available  today.  Tlie  heart  of  these  sys- 
tems will  be  a real-time,  automated  command  and  control  system.  CSC  as  a prime 
contractor  to  the  Navy  has  developed  the  operational  software  for  this  advanced  sys- 
tem based  on  a prototype  system  developed  by  another  contractor. 

The  mission  of  the  DD-963  class  destroyers  is  to  operate  offensively  in  the  presence 
of  air,  surface  or  subsurface  threats  as  an  escort  for  strike  forces  or  antisubmarine 
warfare  (.\.S\V)  aircraft  carriers.  In  addition,  it  is  designed  to  seek  out  and  destroy 
enemy  submarines.  The  system  is  also  capable  of  providing  friendly  aircraft  control 
during  .-\SW  patrol  and  surveillance,  and  search  and  air  rescue. 

2.4.1  System  Requirements 

The  DD-963  project  is  a time  and  materials  contract  with  the  Fleet  Combat  Direction 
Support  Services  Activity  (FCDSSA)  at  Dam  Neck,  Virginia.  In  May  1973,  FCDS.SA 
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requested  CSC  to  assume  responsil)ility  for  two  activities  associated  with  DD-963 
development,  'llie  first  activity  was  to  monitor  Litton  Industries  production  of  a 
prototype  version  of  the  DD-OdS  command  and  control  system  known  as  Model  III. 

In  this  task  CSC  reviewul  the  computer  program  design  specifications  (CPDS),  sub- 
program design  documents,  program  test  plans  and  program  test  jmocodures.  Also, 
CSC  personnel  witncssc-d  all  levels  of  computer  program  testing.  Tlie  objective  of 
the  second  activity  was  to  upgrade  the  prototype  version  of  the  command  and  control 
system,  lliis  involved  developing  a totally  new  system,  designated  Model  IV.  Tlie 
following  subtasks  were  completed  for  the  Mode!  IV  development: 

• llevise  Computer  Program  Performance  Specifications  (CPPS) 

• Write  CJdkS  for  this  model 

• Modify  the  real-time  executive  as  re-quired  by  the  new  CPPS  and  CPDS 

• Code,  debug,  test  and  integrate  the  application  programs 

• Produce  system  test  procedures  and  assist  the  Navy  in  conducting 
program  acceptance  and  system  integration  testing 

It  was  the  second  activity  associated  with  development  of  the  DD-9G3  command  con- 
trol system  which  was  evaluateti  under  this  study.  Die  MPI’  research  was  devoted 
exclusively  to  the  system  development  efforts  for  Model  IV  of  the  DD-963  system. 

2.4.2  Development  Environment 

Tlie  DD-963  development  environment  had  a very  positive  impact  on  the  conduct  of 
system  development  activities.  The  project  technical  staff,  v.hich  averagexi  30  people 
through  the  life  of  the  project,  was  located  on  site  at  Dam  Neck,  Virginia.  These 
personnel  averaged  10  years  data  processing  experience  and  4(i%  had  directly  applic- 
able prior  experience,  includefi  l'YK-7  hardware  and  CMS-2  langTJage  experi- 

ence. Half  of  the  staff  were  senior  members  of  the  technical  staff  or  higher,  while 
73%  of  the  staff  possessed  college  degrees.  Ibe  average,  annual  personnel  turnover 
rate  was  only  13.2%. 

System  development  took  (jlaee  over  a period  of  two  and  a half  years  at  a cost  to  the 
customer  of  approximately  S3..oM.  The  software  system  developed  under  this  con- 
tract consistul  of  14  morlules  comprising  2.aOK  lines  of  Cfxie.  'Hiese  modules  form 
the  ship  operational  program  (S(;iT  whicli  is  an  i;len»ent  of  the  Naval  Tactical  Data 
System  (NIDS)  cotunvand  and  control  system  supporting  the  .\S\V  mission  of  DD-963 
class  destrovers.  Processing  is  handled  by  an  .\N 'l'YK-7  computer  system  con- 
figured with  three  central  i)ioccssor  units  (CPUs),  two  input/output  channels  (lOCs) 
and  ten  memory  batiks.  .\  fourtli  CPU  runs  with  one  IOC  and  two  memory  banks  as 
a reserve  processor.  .Sclnxlul  ing  of  the  modul  es,  coreallocation,  timing,  etc.,  is 
handled  by  the  real-time  executive  program. 

Ihe  primary  out|)ut  ilevice  is  a Hughes  ilisplay  console,  which  displays  a radar/ 
symbology  presentation  and  ampl ifv ing  dat:i.  'lYack  symbology  for  air,  surface  and 
subsurface  tracks  will  be  displayui,  with  modifiers  showing  velocity  and  direction, 
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assignment  or  engagement  status,  etc.  Also  special  points  such  as  reference  points, 
sonobuoys  and  electronic  warfare  fixes  will  be  shown. 


( 

) 

i 

I 


L 


2.-1. 2 I^roject  History 

Initial  development  of  the  DD-963  command  and  control  system  was  begun  by  Litton 
Industries.  Litton  produced  a prototype  version  of  the  system,  known  as  Model  in, 
which  is  now  operational,  at  sea.  Following  production  by  Litton  of  the  computer 
program  performance  specifications  (CPPS)  for  Model  III  of  the  system,  FCDSSA 
reciuested  that  CSC  axpand  its  work  force  at  Dam  Neck  to  include  two  activities  con- 
nected with  DD-963.  CSC  had  a project  office  and  staff  at  Dam  Neck  working  on 
other  related  development  tasks  at  that  time.  As  discussed  earlier,  CSC  assumed 
the  role  of  program  monitor  for  the  balance  of  Model  III  development.  Concurrent 
with  this  activity  CSC  assumed  responsibility  for  development  of  Model  D’  of  the 
DD-963  system  beginning  with  rewriting  the  computer  program  performance  speci- 
fications (CPPS),  Following  this  CSC  proposed  a three  year  computer  program 
development  schedule  for  Model  IV,  but  the  Navy  specified  a two  year  requirement, 
which  CSC  accepted. 

'Ibe  CSC  project  office  at  Dam  Neck  then  expanded  its  staff  and  began  the  task  of  re- 
writing the  computer  program  design  specifications  (CPDS).  As  work  began  it 
became  obvious  that  the  CPPS  would  have  to  be  revised  in  oi-der  to  more  accurately 
reflect  modified  performance  requirements.  After  the  CPPS  was  revised,  the 
CPDS  was  rewritten  and  the  threads  for  the  new  CPDS  were  developed.  Threads 
were  employed  at  this  time  to  verify  the  design  to  performance  relationship  within 
the  system.  Once  the  detailed  design  was  coinpleted  the  code  and  checkout  activity 
began . 


During  the  ccxle  and  checkout  phase  the  DD-963  staff  used  the  builds  concept  to  es- 
tablish system  development  milestones.  By  combining  functionally  related  threads 
into  groups  known  as  builds  it  was  possible  to  define  an  incremental  de\-elopment 
schedule  for  system  capabilities.  Production  responsibility  for  each  build  was 
assigned  to  an  individual  task  (build)  leader.  He  assigned  coding  tasks  and  monitored 
progress  through  formal  demonstration  of  the  system  capabilities  represented  b\-  that 
build.  This  demonstration  was  a formal  presentation  to  the  customer  of  software 
functions  which  satisfied  specific  system  performance  requirements.  Build  mile- 
stone achievement  through  demonstrations  is  recoi’ded  in  Table  2-2,  'Ibis  informa- 
tion was  extracted  from  the  Threads  data  base.  Demonstrations  were  conducted  by 
an  independent,  project  test  grou)i  assisted  by  Navy  personnel . 

System  test  and  integration  requirements  were  satisfied  by  the  build  demonstrations. 
Using  this  "build  a little,  test  a little"  concept  the  system  was  complete  and  ready 
for  installation  when  the  final  build  was  delivered.  'Ibe  MPP  analysis  did  not  axtend 
into  the  installation,  and  operation  and  support  phases  of  the  MPP  iiroject.  These 
activities  were  not  a part  of  the  time  and  materials  contract  for  DD-963, 
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Tabic  2-2.  BUILD  DEMONSTRATIONS 


BUILD 

NAME 

DATES 

SCHEDULED  ACTUAL 

COMMENTS 

A 

12/6/74 

12/6/74 

B 

2/24/75 

3/10/75 

System  Hardware  Modification 
Development  Computer  Out  One  Week 
Threads  Added  to  Build 

C 

4/21  '75 

4/14/75 

D 

6/16/75 

6/27/75 

Hardware  Mock-Up  Problems 

Computer  Timing  Problems 

Software  Integration  Task  Underestimated 

E 

7/22  /75 

7/22/75 

F 

9/8/75 

9/22/75 

Development  Computer  Out  One  Week 
Peripheral  Simulator  Not  Fully 
Operational 

Job  Turnaround  Time  Excessive 

G 

11/24/75 

Rescheduled  IXie  to  Nonavailability 
of  Supporting  Hardware 

H 

1/15/76 

11/24/75 

Replaced  Build  G in  the  Demonstration 
Schedule 

G1 

3A  76 

3/3/76 

Rescheduled  as  Noted  .^bove 

G2 

3/1/76 

3/3/76 

Reschedule  as  Noted  Above 

G3 

3/22/76 

4/9/76 

Rescheduled  as  Noted  .■Mwvc 

I 

3 '22  /76 

4/15/76 

Compiler  and  Hardware  Mock-Up  Problems 

n 

3/22/76 

4/15/76 

Compiler  and  Hardware  Mock-Up  Problems 

5/19/76 

5/19/76 

5/19/76 

5/19/76 

K 

6/15/76 

7/25/76 

Restructuring  Build  Components 
Mock-Up  Availability  Problems 

L 

10/1/76 

Delayed  at  the  Request  of  Customer 

LI 

10/1/76 

Delayed  at  the  Request  of  Customer 

r 


[ 
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2.4.4  Project  MPP 

TTie  DD-963  project  staff  employed  a variety  of  management  and  technical  methods 
and  tools  to  aid  in  the  development  of  the  command  and  control  system  software.  The 
following  is  a list  of  those  MPP  identified  during  the  research  of  this  project; 

• Top  Down  Design 

• TTIRE/XDS 

• Threads  Management  System 

• Builds 

• Build  Leader 

• Structured  Modular  Programming 

• Chief  Programmer 

• Peer  Group  Programming 

• Support  Programs 

• Verification  Procedures 

• Independent  Test  and  Evaluation 

• Progressive  Testing  and  Integration 

• Software  Configuration  Management 


The  following  discussions  will  highlight  the  impact  of  each  of  these  MPP  on  this  pro- 
ject including  the  benefits  of  their  use. 

2.  4. 4.1  Top  Down  Design 


Top  down  design  facilitated  the  consistent  and  complete  definition  of  a system  archi- 
tecture which  satisfied  all  performance  requirements.  This  design  technique  was 
most  effective  for  DD  -963  because  of  the  high  degree  of  subsystem/module  inter- 
action required  by  this  real  time  system.  Definition  of  a system  in  a hierarchical 
manner  facilitates  the  positive  identification  of  system  functional  relationships  and 
assures  that  internal  data  flow  will  be  optimized.  For  the  DD-963  command  and 
control  system  there  were  a number  of  diverse  operational  capabilities  which  had  to 
be  linked  through  a single  executive  to  a master  display/control  subsystem.  This 
was  a complex  design  problem  with  the  level  of  intermodule  message  traffic  as  a 
paramount  consideration.  The  functional  capabilities  had  to  be  designed  so  that  they 
maintained  their  unique  characteristics,  and  yet  communicated  efficiently  with  each 
other  and  the  display /control  subsystem. 

For  DD-963,  top  down  design  was  also  compatible  with  the  use  of  threads  for  design 
verification.  The  threading  process  involves  decomposition  of  the  system  design 
proceeding  from  high  to  low  level  processes.  Thus,  threads  definition  logically 
followed  the  design  methodology/  applied  to  the  DD-963  system.  Top  down  design 
and  threads  were  compatible  with  the  modular  system  architecture  necessary  for 
this  real  time  system  to  provide  the  flexibility  required  in  meeting  the  dynamic 
operational  recjuirements  placed  on  the  diverse  components  of  a command  and  control 
system,  a modular  system  is  the  best  design. 
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The  top  down  desijjn  metl)ixlolo}>v  would  benefit  any  system  which  can  he  specified  to 
operate  with  multipb/  functional  levels.  For  example,  if  an  executive  function  will 
exist  which  exerts  operational  control  of  system  subfunctions  then  top  down  design 
would  be  beneficial . (m  tlie  other  hand,  where  a system  is  composed  of  autonomous 
functions  with  limited  intorfunction  communication,  top  down  design  \w3uld  not  be 
called  for.  In  fact,  it  might  be  counteri»'o<luctive  to  attempt  a top  down  system 
design. 


2. 4. 4. 2 'niKKADS 

niUEADS  is  a discipline  dcvelupcd  liy  CSC  which  may  be  used  to  map  internal  system 
processes  or  operations.  A thread  reiirescnts  a sequence  of  events  or  discrete  opera- 
tions that  lead  to  satisfying  a specific  function  within  a system.  This  sequence  en- 
compasses one  process  from  stimulus  to  response  and  may  be  defined  at  any  functional 
level,  dluis,  threads  may  be  specified  at  the  system,  subsystem  or  program  levels, 

'rite  DD-9G3  staff  threaded  that  system  during  the  design  phase  of  system  develop- 
ment, Tliese  threads  were  used  to  verify  the  design  to  the  requirements  as  the  CPDS 
was  being  developed.  In  this  way  design  flaws  were  detected  early  when  there  was 
more  time  to  make  design  adjustments  without  significantly  impacting  system  develop- 
ment. These  threads  assurai  that  the  performance  requirements  as  specified  in  the 
CPPS  had  been  translatcxl  in  a consistent  manner  into  system  design. 

This  method  of  system  definition  also  assured  that  module  interfaces  were  compatible. 
Later,  during  the  cvxlc  and  checkout  phase  the  threads  were  used  to  monitor  develop- 
ment progress.  The  'Ifireads  Management  System  (TMS)  provided  the  vehicle  for 
reporting  and  displaying  threads  completion  data.  This  provided  management  visi- 
bility into  the  current  status  of  sy.stem  development.  Finally,  test  scripts  were 
developed  using  threads  to  specify  the  processes  to  he  tested  and  to  identify  the  re- 
quired stimuli  and  responses. 

'niRE/VUS  is  a design  verification  tool  with  wide  application.  It  is  a discipline  which 
provides  a straightforward  metlaxl  for  rel ating  design  to  requirements.  As  a result 
of  this  verification  process  s^■stenl  design  flaws  are  more  likely  to  be  detected  early, 
before  crxJing  begins.  Pv  employing  the  'llireads  Management  System  with  the  threads 
data,  the  management  and  the  cmitonier  are  given  good  visibility  into  the  development 
process.  ’Ihrough  TMS,  m.inagcment  is  able  to  develop  greater  flexibility  in  re- 
sponding to  rcfiuircments  changes.  TMS  provides  the  types  of  reports  which  may  be 
used  to  develop  change  impact  asse.ssment.  niilEADS  is  an  excellent  management 
tool  for  n.anagers  at  all  levels. 

2.4,4.d  threads  .Management  .System 

The  Ihrcads  Management  System  ('I’MSj  is  a software  system  developed  by  CSC  that 
can  provide  automated  threads  data  management  services  to  threads  users.  With 


this  system  users  have  access  to  threads  analysis  and  progress  information  that  may 
be  displayed  in  a variety  of  formats. 

■me  data  base  for  the  TMS  as  used  on  the  DD-963  project  contained  identifying  infor- 
mation similar  to  that  described  for  AEGIS  in  Section  2. 2. 4. 4 of  this  report.  This 
information  was  gathered  and  entered  into  the  TMS  data  base  by  a member  of  the 
configuration  managent  staff.  Updates  were  performed  regularly,  usually  on  a 

weekly  basis. 

Ihe  DD-9G;?  staff  used  the  TMS  to  provide  visibility  to  the  customer  during  the  code, 
test  and  integi’ation  activities  for  all  subsystems.  The  customer  was  very  concern^ 
about  receiving  current  information  on  the  true  status  of  system  development.  . 
reports  provided  an  excellent  vehicle  for  satisfying  this  requirement.  For  DD-963 
the  TMS  was  used  in  essentially  the  same  way  that  it  had  been  used  on  the  AEGIS  pro- 
ject Tlie  primary  difference  between  the  two  TMS  applications  was  in  the  level  ot 
usa<^e.  Tlie  DD-963  staff  relied  on  the  TMS  reports  as  the  mainstay  of  their  manage- 
ment reporting  system.  The  customer  was  briefed  regularly  and  frequently  on  pro- 
duction progress'.  Both  the  DD-963  staff  and  the  customer  depended  very  heavily  on 
the  TMS  reports.  These  reports  were  readily  accepted  by  the  customer  because  of 
their  clarity  and  accuracy.  By  using  the  TMS  and  its  reports  the  DD-963  staff  was 
able  to  maintain  management  control  of  the  project  and  develop  an  excellent  working 
relationship  with  the  customer. 


2. 4. 4. 4 Builds 

The  build  concept  is  a natural  corollary  to  THREADS.  Threads  represent  distinct 
processes  within  a system.  By  combining  related  processes  into  groups  which 
represent  demonstratable  system  functions,  builds  are  created.  A build  represents 
a specified  system  capability  such  as  target  tracking,  noise  monitoring,  data  update, 
etc.  Tlireads  do  not  have  to  be  defined  for  a system  in  order  to  use  the  builds  con- 
cept. Any  system  capability  which  may  be  uniquely  identified  may  be  specifi^  as  a 
build.  All  software  routines  supporting  that  capability  become  a part  of  that  build. 


Buih's  were  created  for  DD-963  by  combining  threads  in  order  to  define  an  incre- 
mental software  production  schedule.  Through  the  builds,  specific  systeni  capabili- 
ties could  be  coded,  tested  and  demonstrated  to  the  customer.  This  provided  the 
desired  customer  visibility  into  the  development  process  and  permitted  incremental 
production  of  each  subsystem.  Visibility  was  provided  through  the  build  milestones 
and  TMS  reporting.  The  customer  was  made  aware  of  the  current  status  of  all 
threads  and  thus  all  builds  in  production.  When  production  problems  caused  delays 
or  external  conditions  halted  effective  program  development  on  a given  build, 
resources  could  be  shifted  to  other  builds  and  the  customer  rapidly  appraised  of  the 

schedule  changes. 


The  use  of  builds  would  be  of  benefit  to  iiny  software  development  task  where  incre- 
mental production  of  system  capabilities  is  planned  and  threads  are  defined.  liuilds 
are  also  useful  in  providing  customer  vdsibility  into  the  development  process  and 
may  be  created  for  that  purfKjse  alone.  A supplementary  benefit  is  derived  from 
the  incremental  production  and  intef^ration  of  system  capabilities.  Tlirough  early, 
phased  integration  of  system  components  many  of  the  massive  problems  associated 
with  integration  of  all  subsystems  at  one  time  are  avoided.  Builds  represent  more 
management  control  and  customer  visibility  throughout  the  development  process. 

2.4.4.f)  Build  Leader 

The  function  of  build  loader  was  created  to  satisfy  the  requirement  for  a team  chief 
to  manage  development  of  each  system  build.  This  responsibility  was  assigned  to  a 
senior  member  of  the  staff  (usually  a Chief  Programmer).  The  build  leader  then 
assigned  individual  ()rogramming  tasks  for  production  of  the  build  capability.  This 
group,  headed  by  the  build  leader,  worked  together  as  a team  until  the  build  was 
demonstrated.  This  team  functioned  in  a manner  closely  resembling  the  Chief 
Programmer  'i'eam  described  in  the  Structured  I’rogramming  Series.  A significant 
difference  was  the  absence  of  a team  librarian  function,  'fliis  function  was  common 
to  the  entire  DD-96.T  project.  The  build  leader  concept  of  operations  proved  very 
successful  for  this  project. 

TTie  build  leader  and  team  conceiit  could  he  very  effective  on  any  project  using  incre- 
mental subsystem  production  and  integration  through  builds.  It  i)i’ovides  a focused 
management  organization  with  coordinated  accomplishment  of  limited  objectives. 

2.4.4. 6 StrucUiftid  Modular  ITogramming 


The  pi'ogramming  technique  used  on  the  DD-9G3  project  involved  strictly  enforced 
coding  conventions  similar  to  structured  programming.  Using  this  structured  modu- 
lar programming,  I)I)-f)(i.T  programmers  developcvi  short  programs  with  standard 
data  names,  indentured  code  structure,  and  limited  use  of  "GO  TX)”  statements. 
Single  entry ''singl e exit  logic  for  these  programs  was  stressed,  however  since  the 
high  order  language  (CMS-2)  did  not  have  all  of  tlu;  structured  programming  con- 
structs it  was  not  possible  to  use  pure  stnictured  code.  The  program  code  gene- 
rated by  project  programmers  was  examined  iiy  the  Chief  I’rogrammers  prior  to 
being  submitted  for  initial  compilation.  This  provided  a degree  of  coding  standardi- 
zation enforcement  and  kept  the  Chief  Programmers  ai)prised  of  program  develop- 
ment progress.  The  effect  of  using  this  programming  approach  was  that  all  pro- 
grams were  easier  to  rcjid  and  understand.  Ti’ansfering  responsibility  for  a particu- 
lar program  was  facilitatal.  However,  the  Chief  Programmers'  work  load  was  quite 
heavy  because  most  production  materials,  including  program  ccxJe,  were  tunneled 
through  them. 


Structurod  modular  profjramming  was  simply  a variant  of  more  conventional  pro- 
gramming techniques.  It  was  used  effectively  on  the  DD-%3  project,  but  is  not 
uniquely  better  than  any  other  structured  technique.  Perhaps  its  greatest  worth  was 
in  the  standarization  that  it  imposed  on  the  program  development  work. 

2. 4.  1.7  Chief  Programmer 

llie  DD-963  staff  organization  included  two  Chief  Programmer  positions.  However, 
there  were  no  teams  associated  with  these  positions  as  in  a Chief  I’rogrammer  Team 
structure  defined  in  the  Structured  Programming  Series.  ITie  UD-963  Chief  Ihro- 
grammers  functioned  more  as  section  managers  than  as  task  oriented  team  leaders. 
Actually,  a Chief  Programmer's  responsibilities  included  those  of  a section  manager, 
team  leader  and  technical  advisor.  The  Chief  Programmer  function  was  critical  to 
the  success  of  the  development  effort.  Each  Chief  Programmer  supervisal  a gi’oup 
of  10-15  programmers  and  analysts.  In  addition  he  was  assigned  build  leader  respon- 
sibilities on  a continuing  basis,  lie  monitored  and  examinetl  program  generation  and 
assistal  in  subsystem  testing.  The  Chief  Programmers  were  the  focus  of  program 
development  efforts.  They  assured  continuity  and  consistency  in  design  to  code 
translation  for  the  entire  DD-963  system.  The  efforts  of  these  two  Chief  Program- 
mers were  complimented  by  the  outstanding  management  support  provided  by  the 
Department  Manager.  He  assured  that  the  Chief  Programmers  were  relieved  of  all 
administrative  responsibilities  and  provided  support  services  such  as  configuration 
control,  independent  testing,  program  library  management,  etc.  Under  these  condi- 
tions the  Chief  Programmers  could  devote  their  full  attention  to  program  develop- 
ment, 

'Ilie  benefits  which  accrued  to  this  project  from  the  use  of  a Chief  Programmer  func- 
tion stem  mainly  from  the  individuals  who  filled  the  two  positions.  In  essence  the 
cliief  i)rogrammers  were  seetion  managers,  Tliey  were  given  more  responsibilities 
than  would  ordinarily  be  given  to  section  managers  and  more  latitude  to  exercise 
their  authority.  Under  these  conditions  the  two  individuals  who  were  chief  program- 
mers, with  the  support  of  the  Department  Manager,  did  an  outstanding  job  in  direct- 
ing the  efforts  of  the  technical  staff.  They  made  the  Chief  Programmer  position  an 
effective  management  technique. 

2.4.  l.s  I’eer  Group  Programming  (Structured  Walkthrough! 

i’cer  group  programming  is  a less  formal  version  of  the  structured  walkthrough  pro- 
cess. ’ihe  DD-963  staff  used  this  technique  for  detailed  design  and  program  code 
review  of  each  i>rogram  developed.  Ihese  re\iews  were  led  by  project  task  leaders, 
usually  the  Chief  [Programmers.  Each  review  committee  was  composed  of  the  ()ri- 
mary  progi’amm ei’'s  peers,  other  staff  programmers  aixl  analysts.  ITiis  group 
would  bench  check  the  design  or  ccxle  in  order  to  identify  logic  and  coding  errors. 
I’rojcct  task  leaders  felt  that  with  this  review  process  test  time  was  less  than  half 
that  which  would  have  been  rcxiuirod  without  the  reviews.  For  DD-963  this  was  a 
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significant  factor  because  of  the  compressed  schedule  under  which  they  were  working. 
The  indirect  benefits  derived  from  these  reviews  were  confirmation  of  the  compati- 
bility of  subsystem  interfaces  and  broadened  staff  understanding  of  subsystem  con- 
trol logic. 

ITie  use  of  peer  group  programming  is  beneficial  as  a less  formal  alternative  to 
structured  walktliroughs.  'ITiey  both  accomplish  the  same  purpose,  but  peer  group 
programming  seems  to  be  suited  to  small,  informal  project  staff  organizations. 

'Ihe  cost  in  manhours  to  conduct  these  reviews  seems  more  than  offset  by  the  savings 
accrued  during  program  testing. 

2. 4. 4. 9 Support  I’rograms 

The  support  programs  used  on  the  1)1)-9G:3  project  played  an  important  part  in  the 
successful  production  of  the  software  system.  Support  progTams  w'ere  credited  by 
task  leaders  witii  supplving  added  capabilities  wliich  assured  that  the  technical  staff 
could  meet  its  production  milestones.  'Ilie  support  programs  such  as  those  pro- 
ducing core  dumps,  program  comj)ilations,  message  traps,  selective  memory  regis- 
ter recordings,  and  flow  chart  documentation  were  tailored  to  conform  to  the  require- 
ments of  this  project,  the  nux-lilications  were  completed  by  the  staff  to  insure  that 
the  programs  satisfied  the  specialized  requirements  of  the  DD-9G3  project.  This  was 
a continuing  process  as  neexis  and  requirements  changed.  Developmait  of  the  support 
software  was  aggressively  pursued  throughout  the  project.  For  this  reason  the 
support  programs  did  what  they  were  supposed  to  do  and  all  members  of  the  project 
technical  staff  wore  able  to  use  them  witli  confidence. 

.Support  iH’ograms  can  be  an  effective  aid  in  promoting  software  development.  It 
appears  from  data  gathered  on  the  DD-963  project  that  they  are  most  effective  when 
created  or  tailored  for  a siiocific  system  development.  The  DD-9G3  technical  staff 
had  confidence  in  their  support  programs  and  used  them  frerjuently.  The  programs 
were  maintained  and  modified  conscientiously.  These  factors  combined  to  produce 
an  aggressive  development  program  whose  success  was  due  in  no  small  part  to  the 
support  software. 

2.4.4.10  \’erificativm  Procedures 

Verification  procedures  as  usal  on  the  I)l)-9(’>3  project  included  all  activities  asso- 
ciated with  confirming  the  consistency  of  computer  program  design  and  code.  Con- 
sistency was  judged  against  the  Computer  Program  Performance  Specification 
(CPPS)  anfl  the  Computer  Program  Design  Specification  (CPDS).  Tlie  confirmation 
process  began  with  an  analvsis  of  tlie  computer  program  design  to  assure  that  the 
requirements  statml  in  the  CPl’S  were  satisfied.  Once  the  program  code  had  been 
developed  the  logic  w as  compared  with  the  design  specifications  to  insure  that  it 
conformcfl  to  the  ('PI)S.  Tlie  key  to  successful  implementation  of  these  verification 
procedures  was  the  use  of  the  threads.  Threads  were  a vital  element  of  the  top 


down  verification  process.  For  bottom  up  verification  the  auto-flowchart  documenta- 
tion was  used.  ITiis  support  program  provided  an  automated  flowchart  generation 
capability  to  the  project  staff.  This  documentation  was  compared  to  the  CPDS  during 
program  verification. 

Verification  procedures  had  a very  positive  impact  on  the  DD-963  project.  It  was 
through  these  procedures  that  the  department  manager  and  the  chief  programmers 
were  able  to  control  and  monitor  the  prcxluction  of  program  code  and  the  demonstra- 
tion of  system  capabilities,  'fhe  procedures  established  a formal  set  of  rules  or 
guidelines  which  governed  the  system  acquisition  process,  'fhey  were  instrumental 
in  insuring  that  each  system  function  developed  was  responsive  to  the  performance 
specifications  and  adhered  to  the  design  specifications. 

lliese  procedures  can  be  implemented  on  any  type  or  size  project,  'fhey  would  be 
most  effective  if  used  in  conjunction  with  ITIREADS  and  builds.  The  procedures 
should  be  adapted  to  project  development  recjuirements  and  implemented  early  in  the 
development  cycle. 

2.-1.1.11  Independent  Test  and  Evaluation 

'ITie  primary  function  of  an  independent  test  and  evaluation  te:un  is  to  develop  test 
scenarios  that  will  ensure  all  system  functions  are  tested  and  that  they  perform  in 
accordance  with  the  requirements  specifications,  llie  group  should  be  organizational 
ly  structured  to  rc()ort  to  the  project  manager  rather  than  the  programmer  manager 
in  order  to  provide  unbiased  evaluation  of  the  test  results. 

Ihc  management  ))ersonnel  on  the  DD-9(i:3  project  felt  the  independent  test  and  evalua 
tion  teams  contributed  significantly  to  the  success  of  the  project.  "Hie  impact  of  this 
approach  is  difficult  to  quantify.  However,  when  the  tester  is  not  involved  in  the 
dc'.'clopment  of  the  programs  he  does  not  test  the  conditions  vhich  were  coded,  he 
tests  the  |)crtormance  as  it  was  designed,  'lliis  approach  is  more  comprehensive 
because  more  time  will  be  spent  on  the  test  script  since  this  is  the  primary  function 
of  the  test  group. 

'llie  testing  on  this  project  was  conducted  with  the  customer  present  and  in  some 
cases  the  customer  personnel  actually  operateii  the  consoles  as  they  would  onboard 
the  ship.  This  teclinique  instilled  confidence  in  the  prcxluct  and  provided  training  for 
the  customer,  as  well  as  confirmed  the  actual  "man  to  machine"  interfaces  that  were 
required  to  execute  the  system. 

An  inde[)endent  test  and  evaluation  team  should  be  used  to  piaxiuce  the  greatest  bene- 
fit to  the  project.  tVhen  this  technique  is  not  feasible,  the  tester  should  be  someone 
not  directlv  involved  in  developing  the  program(si  being  tested.  It  is  also  recom- 
mended the  test  plans  and  procaiures  be  developtd  et.rly  in  the  devel opment  cycle 
and  baselined  and  controlled  under  the  Same  conditions  as  the  design  document. 


When  possible,  the  customer  should  ulso  be  involved  in  the  testinjf  in  order  to  insure 
that  he  understands  what  is  involved  in  the  operation  of  the  system  as  early  ns  feasi- 
ble. 

2.4.4.12  Prof^ressive  Testing  and  Integration 

Progressive  testing  is  the  process  of  verifying  subsystem  capabilities  on  an  incre- 
mental basis  as  the  system  is  being  developed,  l^or  the  T)D-963  project  this  tech- 
nique provided  excellent  visibility  for  the  customer  during  the  life  of  the  project. 

By  integrating  these  capabilities  as  they  are  tested  the  massive  integration  problem 
caused  by  attempts  to  integrate  all  subsystems  at  one  time  is  avoided.  The  pro- 
gressive testing  and  integration  process  also  facilitated  detection  of  subsystem 
interface  errors  earlier.  In  conjunction  with  testing  new  system  capabilities  the 
previously  integrattxi  system  capabilities  were  tested  through  regressive  testing. 

This  insured  that  the  newly  integrated  software  did  not  adversely  affect  the  existing 
system  functions,  llie  use  of  this  technique  along  with  other  MPP  permitted  the 
DD-963  staff  to  provide  the  customer  with  a current  status  report  of  system  progress 
and  to  demonstrate  system  capabilities  as  they  were  developed. 

'Ihe  total  effects  of  i)rogressive  testing  and  integration  on  the  DD-963  project  cannot 
be  determined  at  this  time  as  they  have  not  reached  the  system  installation  phase, 
however  based  on  the  results  of  other  projects  using  this  technique  the  benefits  will 
outweigh  the  effort  expended. 

llie  use  of  this  techniciue  will  benefit  any  project  where  communication  between  sys- 
tem components  is  rec|uirul.  When  used  in  conjunction  with  other  MPP  such  as  the 
TIIREADS/Ruilds  concept  the  cost  of  applying  the  tool  is  minimal  and  the  benefits 
derived  make  it  very  cost  effective. 

2.4.4.13  Software  Configuration  Management 

Software  Configuration  Management  (SCM)  is  the  process  of  controlling  the  software 
system  configuration  after  a specific  configuration  has  been  declared  to  be  a base- 
line position.  'Ihe  baseline  position  establishes  program  structure  for  formal  test- 
ing and  demonstration.  Once  the  baseline  is  established,  it  is  the  responsibility  of 
the  SCM  staff  to  control  program  nudification  and  prevent  unauthorized  changes. 

Ihe  I)I)-9(j3  SCM  staff  consisted  of  one  person  who  produced  all  system  version 
positions  for  build  demonstrations  and  who  control  ed  all  changes  to  current  system 
versions.  lie  was  aided  by  the  Chief  Programmers  who  reviewed  program  code  and 
monitored  production  of  new  program  versions.  However,  these  new  program  ver- 
sions were  transferred  to  the  system  file  without  recompilation  because  of  manpower 
and  production  facility  constraints,  .\pparently  this  did  not  create  any  problems 
during  testing  and  integration,  but  it  was  a potential  problem  area.  Overall,  SCM 
was  vigorously  applied  on  the  DI)-9(33  project  and  contributed  to  the  successful 
demonstration  of  all  system  builds. 
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An  SCM  function  is  usually  associated  with  all  projects,  llie  size  of  this  staff  func- 
tion should  be  determined  by  the  size  of  the  project.  The  configuration  control  plan 
should  be  developed  during  the  system  design  phase  and  the  plan  should  be  enforced 
to  ensure  the  integrity  of  the  system  configuration  throughout  the  life  cycle  of  the 
project.  TTie  benefit  derived  from  SCM  is  directly  proportional  to  the  conscientious- 
ness with  which  it  is  applied. 
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SECTION  3 - MPP  ANALYSES 


3.1  OVER  VIEW 

nie  modern  programming  practices  that  were  identified  during  the  research  of  CSC 
projects  have  been  introduced  as  part  of  the  project  descriptions.  Those  discussions 
focused  on  how  the  practices  were  used  on  each  project  and  wdiat  benefits  were 
derived  from  their  use.  In  this  section  each  practice  will  be  analyzcxl  from  the  stand- 
point of  its  impact  across  the  projects  studied.  In  this  way  a more  comprehensive 
and  definitive  analysis  can  be  developed. 

Tlie  format  of  these  individual  programming  practice  analyses  was  selected  to  facili- 
tate the  kind  of  critical  review'  which  will  produce  a clear  definition  of  the  MPP  value, 
'rhese  discussions  have  been  developed  around  four  topics.  First,  each  practice  w'ill 
be  described  as  completely  and  accurately  as  possible.  The  descriptions  will  include 
all  forms  of  each  pi-actice  that  w'ere  employed  on  the  projects.  Next,  the  effective- 
ness of  each  practice  within  applicable  development  phases  will  be  discussed. 
Effectiveness  will  be  measured  using  quantitative  data  such  as  error  generation  rates, 
code  production  rates,  etc.  when  possible.  Also,  qualitative  assessments  w'ill  be 
used  in  lieu  of  or  to  supplement  quantitative  data.  Computer  program  life  cycle 
phases  based  on  AFM  800  series  definitions  will  be  used  in  the  discussions  of  MPP 
effectiveness.  Then,  a comparison  will  be  made  of  the  impact  of  each  practice 
across  each  of  the  projects  where  it  was  used.  This  comparison  will  consider  how 
each  practice  was  employed  on  the  projects  and  how  the  MPP  influenced  each  individ- 
ual project.  Finally,  recommendations  for  use  of  these  MPP  on  future  projects  will 
be  pro(X)sed.  These  recommendations  will  be  based  on  the  benefits  and  costs  asso- 
ciated with  use  of  the  MPP. 

3.2  TOP  IKAVN  DESIGN 

Top  down  design  is  the  technique  of  design  definition  which  proceeds  from  tlie  highest 
functional  level  to  the  lowest  1 evel  within  the  system.  This  process  is  accomplished 
by  defining,  at  the  highest  level,  a functional  requirement  to  be  performed,  then 
refining  that  definition  in  terms  of  its  functional  components  at  the  next  lowest  level 
and  continuing  the  process  until  the  most  elejiientary  level  is  reached.  It  may  be 
disti..'guishod  Irom  bottom  up  design  where  functional  components  at  the  lowest  level . 
i.e.  routine,  in’ogram,  tnodule,  are  defined  first.  TTiese  eoni]ionents  are  then  1 inked 
through  further  system  definition  to  succeedingly  higher  le\els  until  the  highest  or 
system  executive  1 evel  is  reached. 

3.2.  I Development  Phases 

This  programming  practice  is  applied  during  the  design  phase  of  the  development 
cycle,  however  tlie  design  th.it  results  will  affect  all  following  phases.  The  value  of 
this  technkgie  stems  from  its  inherent  abilitv  to  jiromote  design  comjil eteness  in 
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each  level  of  the  hieranhv  hy  forcin'?  each  level  to  be  loj?ically  complete  within  itself. 
The  interfaces  between  the  system  functional  components  are  identifier!  and  can  be 
demonstraterJ  after  the  top  level  program  has  been  developed  through  the  use  of 
dummy  execution  routines,  llus  process  leads  to  detection  of  integration  problems 
early  in  the  development  cycle  when  they  can  be  corrected  at  a much  lower  cost. 

llie  three  CSC  projects  reviewed  all  used  top  down  design  and  realized  benefits  in 
both  the  management  and  technical  areas.  Itiis  technique  supports  pi'oject  manage- 
ment through  its  systematic,  hierarchical  design  process  and  functionally  consistent 
task  and  development  milestone  definition.  'I'op  down  design  promotes  functional 
traceability  as  system  ro(|uirements  are  translated  into  system  components.  'Ihe 
hierarchical  apj)roach  insunus  clearly  defined  functional  paths  as  progressively  lower 
level  system  components  are  defined.  This  design  process  promotes  optimization 
because  functional  paths  are  allocated  based  on  the  logical  progression  of  system 
ret]uirements  down  to  the  lowest  level . Top  down  design  supports  system  mainte- 
nance into  the  operation  and  support  phase  through  the  modular  definition  of  system 
components.  Software  modules  evolve  as  functionally  complete  components  w'hich 
facilitates  isolation  of  m-il functioning  system  elements,  'niese  mixiules  also  sup- 
port system  portability  bv  providing  a capability  for  selective  implementation  of 
system  functions  based  on  hardware  or  operational  limitations. 

3.2.2  MPT  Comparison 

There  were  no  researcii  limlings  to  substantiate  that  top  down  design  was  more  effec- 
tive on  any  one  of  the  projects  than  another,  lire  project  which  did  not  thread  the 
design  did  encounter  some  problems,  one  being  a redesign  of  the  system  due  to  a 
major  concept  change,  huwcwer  the  in;oblems  could  not  be  linked  to  the  original  de- 
sign of  the  software.  The  top  ilown  design  and  modular  approach  to  the  development 
of  this  system  a.«sistui  in  .salvaging  a minor  part  of  the  original  ccxle. 

3.2. 3 Recom mendation.s 

Top  down  design  is  recommenda!  for  a software  development  where  the  total  software 
system  is  to  be  de\  eloj-otl . In  some  cases  where  a major  part  of  the  system  might 
be  "off  the  shelf  software  packages  usal  at  lower  levels  of  a system,  it  might  be 
advaiitageious  to  use  a bottom  up  design.  When  using  existing  software  packages  it 
could  1)0  easier  to  divsign  a system  to  fit  the  packages  rather  than  changing  the  pack- 
ages to  fit  the  system.  'Ih.'  top  down  approach  involves  a more  detailed  functional 
design  than  is  {)roduce«.l  by  other  design  techniques.  This  means  greater  initial  cost 
in  desi;?n  devclopnu^nt  but  this  is  more  than  offset  by  later  savings  in  manhours 
during  detail  oil  design. 

Tlie  benefits  discussed  in  biis  report  and  the  completeness  normally  found  in  the 
architecture  of  a top  down  de.siyn  are  the  major  factors  used  for  the  basis  of  this 
recommendation . 
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3.3  TllUlwVi:>S 


The  niUE.'XDS  methodology  is  a design  verification  tool.  A thread  represents  a 
sequence  of  events  that  contributes  to  the  satisfaction  of  a specific  system  function. 
Tliis  sequence  is  an  internal  process  and  may  he  defined  at  any  functional  level.  A 
single  thread  demonstrates  a discrete  series  of  operational  activities  from  stimulus 
to  response. 

In  threading  the  system  design  the  following  is  accomplished  for  each  operational 
process: 

• Isolate  inputs  available  (stimuli) 

• Identify  outputs  to  be  produced  (responses) 

• Determine  processing  required  to  produce  each  output  from  the 
inputs  (processes) 

• Define  the  sequential  processing  path  from  each  stimulus  to  its 
response  (thread) 

A complete  set  of  threads  defines  all  processing  rec|uired  to  satisfy  all  system  func- 
tional requirements,  'llius,  threading  a system  design  is  a process  employing  a 
discipline  that  is  external  to  the  system  to  verify  design  consistency  relative  to  re- 
quirements. 'nillE.-XDS  is  a uniquely  functional  approach  to  design  verification  that 
may  be  used  effectively  in  other  roles  such  as  test  plan  development  and,  in  conjunc- 
tion with  the  llu’cads  Management  System  (TINIS),  management  control. 

3.3.1  Development  Phases 

Tlie  three  projects  which  were  subjected  to  IVIPP  evaluation  reflect  three  distinct 
management  philosophies  with  respect  to  the  use  of  'lllREADS.  In  the  case  of  .'\EGIS, 
one  subsysteni  was  threadai  and  those  threads  were  used  lo  \-erifv  design  specifica- 
tions and  to  monitor  software  development  progTess.  llie  nUDENT  software  system 
was  not  threaded.  For  DD-963,  the  entire  software  system  was  initially  threaded 
during  the  design  phase,  and  in  conjunction  with  the  llireads  Management  System 
threads  were  aggressively  used  throughout  the  period  of  system  development.  It  is 
worth  noting  that  both  .AEGIS  and  DI)-9(>3  were  successful  projects  in  that  they  were 
completed  on  time  and  within  budget.  IKLDEXT,  on  the  other  hand,  encountered 
serious  development  problems  and  as  a result  was  not  as  successful. 

Threads  are  most  effective  when  they  are  introduced  earl_\-  in  the  system  acquisition 
process.  'Ilircads  employment  should  be  definul  within  the  project  management  plan 
as  an  integral  part  of  the  total  management  tdfort.  System  level  thrca<is  may  be  used 
in  the  iterative  [irocess  of  evaluating  alternative  :irchitecturc  conceiits  preparatory 
to  selecting  the  optimum  architecture  for  system  development.  ITiese  system  level 
threads  must  be  defined  based  on  system  rcKfuirements  specification.  As  system 
develo|)ment  continues,  lower  level  threads  sui)port  activ  ities  such  as  system  and 


subsystem  design  verification,  interface  definition,  test  plan  development  and  system 
integration. 

For  the  DD-90:?  project,  analysis  and  design  phase  activities  includof^l  revision  of 
Computer  i’rogram  Performance  Specifications  (CPPS)  and  writing  the  Computer 
Program  Design  Specifications  (CPDS).  During  these  phases,  CSC  personnel  thread- 
ed the  system.  It  was  necessary  to  extensively  revise  the  CPPS.  The  threading  pro- 
cess was  carried  on  in  conjunction  with  writing  the  CPDS.  This  provided  an  extreme- 
ly effective  vehicle  for  linking  the  design  document  to  the  performance  specifications. 
Developing  threads  during  the  analysis  and  design  phases  assured  more  effective  use 
of  available  manpower  in  addition  to  improving  the  quality  of  the  design.  Also  these 
threads,  developa.1  earl>'  in  the  acquisition  cycle,  continued  to  enhance  the  system 
development  process  throughout  later  phase.s. 

nie  .'XFCiIS  Commiand  and  Decision  subsystem  was  not  threaded  until  after  detailed 
design  was  established.  ITireads  were  defined  after  the  fact  to  verify  the  design 
specifications.  This  was  tne  first  use  of  the  ITIRKADS  methodology'  and  thus  in- 
volved a period  of  learning  for  those  developing  the  threads.  Nevertheless,  they 
were  effective  in  design  verification  and  continued  to  be  used  in  support  of  later 
development  phase  activities. 

During  the  co<le  and  ciieckout  phase  for  both  the  AEGIS  and  DD-963  projects,  threads 
were  used  by  project  managers  to  monitor  and  report  system  development  status, 
'fhread  development  status  reported  through  the  TIMS  is  a reliable  indicator  reflecting 
progress  toward  achieving  project  milestones.  Completion  of  specific  threads  within 
each  build  was  easily  recorded  and  poriixlicaUy  tiiis  data  would  be  reported  in  sum- 
mary form  for  project  monitors  and  managers  to  review.  This  provided  all  con- 
cerned personnel  with  current,  easily  understood  project  status  information. 
Recording  the  thread  data  rcijuircs  the  expenditure  of  some  manpower,  but  not  a 
significant  amount.  For  instance,  on  the  DD-9G:1  project  approximately  50%  of  one 
person's  time  on  a weekly  basis  was  dovotal  to  maintaining  a threads  data  base. 

Test  and  integration  phase  activities  for  the  13D-9G.'l  project  included  developing  and 
executing  test  plans  built  around  the  system,  subsystem  and  module  threads,  llireads 
providal  the  basis  for  the  stimulus  /response  type  material  includeti  within  the  test 
scripts.  'I’his  was  a natural  outgrowth  of  the  development  process  to  this  point.  Test 
team  members  could  be  assured  of  thoroughly  exercising  all  software  functions  be- 
cause of  the  one-to-one  corrcS))ondence  with  system  design  elements.  Eater,  the 
integration  of  system  components  was  facilitated  for  both  DD-0(j3  and  .'XEGIS  person- 
nel because  of  the  close  correlation  among  all  system  components  achieved  through 
threails  definition.  'Ilire.ids  providwi  the  framework  for  positive  control  of  the  test 
and  integration  phase. 

'Dicre  is  little  l•csearch  data  to  support  positive  conclusions  concerning  threads  effec- 
tiveness in  the  installation,  and  operating  and  support  phases.  Research  was  limited 
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to  proinstalUition  dovolopment  for  all  the  projocts.  It  seems  reasonable  to  assume, 
however,  that  the  operating  and  support  phase  would  benefit  from  tlireads  doeumen- 
tation.  With  the  system  functions  clearly  defined,  it  should  be  easier  to  make  the 
modifications  and  functional  adjustments  normally  associated  with  this  phase. 

3.3.2  MPP  Comparison 

'fhe  AEGIS  and  DD-9G3  project  staffs  made  effective  use  of  the  system  or  subsystem 
threads,  'llireads  were  emi^loyed  with  txjual  success  on  both  projects  as  a design 
verification  tool.  I.ater,  through  the  lATS,  threads  became  an  invaluable  aid  in 
maintaining  management  and  customer  visibility  of  software  prfxluction  status, 
nireads  status  reports  were  considered  by  the  staffs  to  be  one  of  tlie  most  current 
and  definitive  indicators  of  production  progress.  For  AEGIS  the  reports  were  pri- 
marily used  for  internal  management  control  and  monitoring.  'Ihe  DD-9G3  staff 
employed  the  reports  internally  and  also  used  them  regolarlv  to  keep  the  customer 
informal.  Once  the  build  demonstrations  and  phased  integration  began  on  the  DD-963 
project  the  threads  played  a dual  role.  They  continued  to  be  usai  to  report  progress. 
In  addition  the  test  group  used  threads  to  develop  and  modify  test  scenarios.  In 
general  the  DD-9G3  staff  seemed  to  employ  threads  a bit  more  aggressively  than  the 
AEGIS  staff  did.  Thus  threads  had  greater  impact  on  the  DD-9G3  project. 

3.3.3  Ilecommendations 

'ITie  decision  to  use  '1TIRE.^\DS  should  be  made  early  in  the  acquisition  life  cycle. 
"ITu'cads  arc  most  effective  when  introducal  early  in  the  analysis  phase  to  support 
the  design  verification  effort.  Once  the  commitment  is  made  to  use  threads  and  they 
are  dcfinal,  their  utility  increases  over  the  life  of  the  project.  In  addition  to  pro- 
viding traceability  directly  from  requirements  to  all  levels  of  design,  threads  sup- 
port effective  management  of  software  development  and  maintenance.  'ITiis  includes 
management  visibility  into  development  progress,  test  scenario  definition  and  e.xecu- 
tion,  and  system  and  subsystem  integration.  niliE/XDS  must  be  consideral  an  im- 
portant element  of  a coordinatal  software  management  plan. 

3.-1  Bin.DS 


A "build”  is  a collection  of  functionally  relatai  processes  that  is  implemented  and 
domonstratal  as  a set.  System  development  employing  builds  includes  early  visi- 
bility of  actual  system  functions,  incremental  availability  of  actual  operating  capa- 
bilities, and  distributed  integration.  By  its  definition,  a build  is  functional.  Be- 
cause a build  is  composal  of  individually  operational  iiroccsses,  the  integration  of 
each  build  into  the  system  supports  progressive,  functional  level  testing.  When  each 
build  is  operational,  it  provides  a capability  that  is  potential  1\  totally  operational, 
rather  than  pieces  and  parts  that  may  be  only  partiallv  operational. 


3.4.1  Development  Phases 


llie  builds  concept  is  uppli«l  in  the  design  phase  to  support  detailed  design  by  identi- 
fying relatal  threads  or  processes  and  combining  them  into  functional  entities. 
Actually,  threads  are  produced  using  the  functional  design  and  builds  are  established 
prior  to  detaiUxi  de.sign.  During  this  same  time  frame  the  development  schedule  is 
determined  and  a com(>lction  date  for  each  build  is  set.  'Ihese  completion  dates  are 
used  as  milestones.  'J'he  benefit  of  this  technique  is  realized  during  the  test  and  in- 
tegration phase.  With  each  build  being  testwi  as  it  is  completed  and  integrated  into 
the  system,  any  interface  problems  can  be  detected  and  corrected  early  in  the  devel- 
opment cycle.  The  other  major  benefits  ficrivcd  from  this  technique  are  an  ability  to 
measure  the  perforniance  of  the  deseloper  throujli  the  system  capabilities  being  per- 
formed during  the  incremental  denumstrations,  as  well  as  assuring  the  user  that  the 
system  functions  do  perform  in  the  manner  in  which  he  had  env'isioned. 

When  used  properly  this  programming  practice  can  be  a very  effective  tool.  By  group- 
ing related  functions  or  processes  together  into  demonstrable  segments,  the  developer 
Can  control  the  system  at  a much  lower  level.  Ihis  technique  promotes  incremental 
development  with  early  integration  and  provides  milestones,  as  well  as,  visibility  of 
progress. 

During  the  development  of  the  .\LG1S  system,  the  need  to  segment  and  build  the  sys- 
tem in  increments  was  realized.  Ihe  threads  technique  was  developed  to  segment 
the  system  and  funi.'tionally  related  threads  were  grouped  into  builds  to  provide  func- 
tional components.  These  tools  provai  very  effective  throughout  the  development 
cycle  and  were  refinerl  and  enhanced  for  use  on  other  projects. 

Tlie  DD-9()3  project  senior  level  personnel  came  from  the  AEGIS  project  and  the 
techniques  that  had  proven  effective  were  brought  along  for  use  on  DD-9f)3.  The 
threads  and  builds  concepts  were  used  and  have  been  very  successful.  The  builds  on 
this  project  were  used  to  demonstrate  functional  capabilities  to  the  customer  (T.S. 
Navy)  who  ref|uircx:l  a high  degree  of  visibility  during  the  development  cycle.  The 
builds  were  set  as  milestones  for  tracking  development  and  were  monitored  very 
closely.  Even  ch.mges  in  the  schedule  of  builds  because  of  unavtiil abil ity  of  hardware 
or  due  to  customer  rcKriuirements  were  made  with  minimum  impact  to  production 
efficiency. 


3 .4.2  MPJ’  ('oniparison 

For  all  three  projects  builds  providcxl  a means  for  expressing  the  functional  decom- 
position of  thedetailal  design.  Thus,  the  |)roject  management  staff  of  each  of  these 
projects  was  able  to  break  down  the  system  to  be  developed  into  functional  capabili- 
ties. Each  system  level  function  became  a build.  'Ihis  apin’oach  led  quite  naturally 
to  the  creation  of  system  development  mil  intones  based  on  builds  production.  Thus, 
a production  schaiule  uas  e.stabl isluxJ  that  was  realistic  in  its  goals  and  flexible 
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enouf^h  to  be  adapted  to  changing  circumstances . Ulien  threads  were  used  to  develop 
the  builds  the  two  MPP  together  became  a [jowirful  management  tool . Cnder  these 
conditions  builds  fostered  the  phased  integration  of  system  capabilities  ns  they  were 
produced.  In  addition,  with  or  without  tlireads,  builds  provided  a medium  for  main- 
taining excellent  management  and  customer  visibility.  'Ibe  build  milestone  schedules 
clearly  defined  the  system  decomposition  and  presented  an  understandable  plan  for 
production  and  integi’ation  of  all  system  capabilities. 

3.4.3  Recommendations 

Builds  are  an  effective  adjunct  to  threads  and  TMS  for  management  control  and  moni- 
toring of  the  software  development  process.  Using  threads  as  elements  of  the  builds 
leads  quite  naturally  to  functional  differentiation  of  the  system.  For  instance  in  a 
real  time,  command  and  control  system  such  as  that  developed  during  the  DD-9G3 
project,  builds  were  defined  for  major  subsystem  functions  - Manual  'lYacking, 
Beacon  Video  Processor,  Underwater  Fire  Control  System,  etc.  Build  milestones 
should  be  established  at  regular  intervals  based  on  program  development  capabilities. 
Again,  using  DD-963  as  an  example,  builds  were  demons tr at txi  (milestones)  on  an 
average  of  every  eight  weeks.  The  schedule  varied  because  builds  varied  in  size 
ranging  from  32  to  542  threads.  The  average  size  build  contained  1G7  threads. 

However,  thread  definition  is  not  a prerequisite  to  the  employment  of  builds.  Builds 
may  be  defined  based  solely  on  functional  decomposition  of  the  system  at  a high  level. 
In  either  instance  builds  provide  a method  for  establishing  dev'clopment  milestones 
and  instituting  incremental  production  and  integration  of  software  subsystems.  The 
build  demonstrations,  where  functional  capabilities  were  tested  to  show  the  user  that 
the  software  satisfied  system  requirements,  also  were  vehicles  for  confirming  that 
previously  demonstrated  capabilities  still  functioned  as  designed.  This  was  accom- 
plished through  regi’ession  testing  which  was  part  of  every  build  demonstration. 
Regression  testing  consisted  of  executing  test  scenarios  from  earlier  build  demon- 
strations. The  test  coordinator  selected  scenarios  that  would  demonstrate  key 
operations  which  were  related  to  the  current  build's  capabilities. 

'fherefore,  builds  are  recommended  for  any  large  software  system  composid  of  mul- 
tiple functional  subsystems.  For  systems  that  are  threadal  the  builds  will  be  a natu- 
ral evolutionary  step  in  system  definition.  The  incremental  development  approach 
which  builds  facilitate  is  a powerful  management  tool  i)arlieularly  for  large,  on-line 
systems. 

3 . 5 TTIRKADS  MANAGEMENT  SYSTEM 

The  TTIRFADS  Management  System  (TMS>  serves  to  automate  and  simplify  the  pro- 
cess of  creating,  uixiating,  interrogating  and  reporting  TTIRF. ADS  information  to 
management.  The  standard  T.MS  output  reports  inel ufie  build  status,  thread  status, 
thread  descriptions,  threads  schetlule  and  build  summarv.  Fxamples  of  these  reports 


are  shown  as  Figures  3-1  through  3-5.  ITie  TMS  operates  on  a variety  of  computers 
and  provides  a wide  range  of  capabilities  depending  on  the  requirements  of  the  user. 

3.5.1  Development  Phases 

llie  TMS  could  be  implemented  at  any  phase  of  the  development  cycle.  The  maximum 
utilization  of  this  tool  is  realized  by  implementing  it  when  the  system  is  threaded  or 
immediately  following  detail ed  system  design.  7he  TMS  can  be  an  effective  tool  in  all 
phases  of  the  program  life  cycle  once  threads  are  defined  if  all  capabilities  of  it  are 
exercised.  J’roject  pi  anning,  configuration  control , monitoring,  testing  and  verifica- 
tion/validation are  all  supported  by  threads.  The  TMS  is  also  applicable  after  the 
development  cycle  as  a configuration  management  and  change  impact  assessment  tool 
during  the  maintenance  phase. 

3.5.2  MPP  Comparison 

llie  effect  this  programming  practice  had  on  the  three  projects  varied  greatly.  The 
ntlDENT  project  was  not  threaded  and  had  no  TMS.  'Ihe  AEGIS  project  was  partially 
threaded  and  the  TMS  was  utilized  as  a monitoring  tool.  'Fhe  DD-OfiS  project  was 
totally  threaded  and  the  'I'MS  was  a major  contribution  in  all  phases  of  the  development 
cycle.  The  DD-')()3  project  used  the  threads  to  verify  the  design  back  to  the  require- 
ments, develop  the  builds,  plan  the  development  sehodu! e,  monitor  the  progress,  and 
develop  the  test  scripts;  and  they  will  be  used  as  a software  configuration  tool  during 
the  maintenance  phase. 

I 

ITie  TlIHEMiS  concei)t  and  the  7’MS  were  conceived  on  the  AEGIS  project  and  were 
used  very  effectively.  However,  the  DD-,063  project  used  the  technique  after  it  had 
been  enlianced  and  the  benefits  derivul  from  the  TMS  during  tliis  project  was  greater 
due  to  the  additional  capabilities.  'Hie  effect  of  this  practice  is  proportional  to  the 
degree  of  usage  on  the  three  projects.  It  had  no  impact  on  'HIIDENT,  it  was  con- 
sidered a \ ery  effective  tool  on  .\EGIS  and  a key  element  on  DD-9fi3  project  in  pro- 
viding management  visibility.  'Hie  TMS  also  supported  other  MPP  used  during  the 
development  of  the  DD-9(i3  software  system. 

3.5.3  llccommendations 

CSC  recommends  the  usage  of  a Tlireads  Management  System,  when  threads  are  im- 
plemented, to  receive  the  maximum  benefits  from  a niHEADS  data  base,  'lliere  is 
some  cost  in\ol\  eii  in  maintaining  and  uidating  the  data  base;  for  instance,  50'J  of  a 
non-senior,  technical  staff  member's  time.  However,  the  TMS  is  an  effective 
management  tool  tlrnt  provides  definitive  data  on  program  de\  elopment  status.  The 
benefits  gained  are  in  improvul  immagement  control  and  visibility. 
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Figure  3-1.  ITireads  Management  System  Retrieval  Report  i 
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This  report  depicts  selected  data  concerning  all  the  nireads  at  a designated  level . \ 
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I 'ITie  ITireud  Sehetlule  Report  indicates  the  status  of  each  Ttuild.  Roth  scheduled  and 

actual  activities  for  each  development  phase  are  shown.  Also  included  are  the  Build 
start  dates  and  a legend  to  assist  the  reader  in  understanding  the  chart. 

1 


3-10 


SUBSVSTtN  LEVEL  THREAOIS) 

project  CtC 

SUBFUNCTION  : SPECIAL  THREAT  SETUP  BUILD  : J 

THREAD  NAME  : COl-001  STATUS  : 0 

DATE  LAST  UPDATE  : 09/27 
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Figure  3-3.  Subsystem  Level  Thread  Report 

This  report  presents  a graphic  representation  of  the  Subsystem  Level  Threads.  The 
boxes  represent  the  subsystem  elements  and  the  processing  required  within  each  ele- 
ment. The  Threads  that  require  more  than  tw'o  boxes  would  display  the  additional 
boxes  directly  below  the  two  shown. 
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PROJECT  C>C  SUSSTSTEH  LEVEL  THREAD  STATUS  •AOC  001 

DATE  12/22/72  TOR  BUILD  C 
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Fi‘;'urc  l>— t.  Subsystem  Level  'Fliread  Status  Report 


llic  Subsystem  Level  'niread  Status  Report  jj;ives  detailed  information  concerning 
each  individual  thread.  The  report  is  produced  for  any  designed  build.  It  lists  each 
thread  by  name  and  shows  the  subsystem  elements  involved  in  each  thread.  The  date 
indicates  when  the  TMS  data  base  was  last  U|x3ated  for  this  thread  information.  The 
status  is  shown  for  each  development  phase  and  by  subsystem  element.  It  refers  to 
that  portion  of  the  clement  that  pertains  to  the  thread  shown.  In  the  example,  the 
CDI  subsystem  element  has  complotal  code  for  the  first  thread,  but  has  only  com- 
pleted design  for  the  sckoikI,  and  so  on. 
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Figure  3-5.  Subsystem  Level  Build  Status  Report 


This  report  is  designed  to  provide  an  overview  of  the  build  status.  It  includes  the 
build  name,  the  number  of  threads  that  are  identified  in  each  build,  and  the  percent- 
age of  completion  for  each  development  phase  (design,  code,  debug,  and  integrate). 
F.ach  thread  has  either  completed  or  not  completed  a particular  phase.  The  overall 
build  status  is  derived  by  the  use  of  an  algorithm  that  gives  "weights"  to  each  of  the 
development  phases.  Hiis  is  necessary  since  the  coding  of  a thread  requiras  less 
effort  than  the  debugging  phase.  A comparison  of  the  schedulai  and  actual  build 
completion  dates  will  give  an  indication  of  the  project  status,  'ilie  sample  shows  a 
Part  A and  Part  R summary.  Each  Part  is  a logical  collection  of  builds  and  repre- 
sents a significant  milestone.  The  analysis  of  a particular  system  would  determine 
whether  or  not  this  particular  feature  is  applicable. 
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3.6  CHIEF  PROGRAMMER 

The  Chief  ProgTammer  is  the  senior  technical  staff  member  assigned  to  a specified 
organizational  level.  For  a project  employing  20  to  30  technical  staff  members  the 
Chief  Programmer  may  direct  the  entire  software  development  effort.  On  the  other 
hand  a project  requiring  a technical  staff  of  100  to  150  personnel  may  use  two  or  three 
Chief  Programmers.  The  span  of  control  assigned  to  any  Chief  Programmer  position 
depends  ver'-  heavily  on  the  technical  and  management  expertise  of  the  individual 
selected  for  the  position.  A Chief  Programmer  is  the  focal  ppint  for  all  software 
development  within  his  organizational  boundary.  He  must  be  expert  in  his  mastery 
of  all  elements  (jf  program  development,  i.e.,  computer  languages,  hardware,  JCL, 
etc.  He  is  a manager  also,  assigning  tasks,  monitoring  progress  and  adjusting  re- 
sources to  maintain  schedules.  Much  of  a Chief  Programmer’s  responsibility  is 
compatible  with  a Section  Manager's.  But,  a Chief  Programmer  fills  a position  of 
broader  responsibility  than  a Section  Manager  as  the  most  senior  or  one  of  the  group 
of  most  senior  technicians  on  a project. 

Hie  Chief  Programmer  does  not  function  in  a team  chief  role  as  described  in  the 
Structured  T’rogramming  Series.  He  has  a task  gi’oup  of  programmers  and  analysts 
involved  in  coordinated  subsystem  development.  There  is  no  group  librarian  or  con- 
figuration management  function.  These  requirements  are  satisfied  by  project  staff 
personnel  who  perform  these  functions  for  the  total  development  effort.  Likewise, 
quality  assurance  and  testing  are  project  functions  and  thus  outside  the  group's 
responsibil  ity . 

3.6.1  Development  Phases 

The  two  Chief  Programmers  for  the  DD-;)63  project  were  assigned  project  staff  posi- 
tions. llieir  basic  responsibilities  did  not  vary  appreciably  from  phase  to  phase 
during  the  project.  They  were  primarily  concerned  with  technical  direction  for 
assigned  software  [iroduction.  'lltey  did  lead  individual  programming  teams  which 
were  assigned  specified  builds  to  develop  and  test,  d'he  department  manager  pro- 
vided management  support  which  freed  the  Chief  Programmers  to  direct  software 
development  with  minimum  distraction.  Overall  this  was  an  c.xtremely  effective  pro- 
ject organization.  It  was  successful  in  jiart  because  the  project  staff  was  small  so 
that  the  span  of  control  tor  each  Chief  Programmer  (10  to  12  personnel)  and  the 
Department  Manager  was  within  manageable  hounds.  .Also,  the  superior  professional 
and  technical  skill  of  both  Chief  Programmers  and  the  Department  Alanager  was  a 
significant  contributing  factor. 

During  all  phases  of  system  de\elopmeat  the  Chief  Programmers  were  focal  points 
for  software  production.  Ihis  began  during  the  analysis  and  design  phases  when  they 
weredeeplv  involved  in  i)reiniration  of  the  I'erformance  and  design  specifications. 
Tliey  also  jiarticipatui  in  the  definition  of  system  threads  during  this  phase.  Later, 
during  the  code  and  checkout  phase  tliis  exiierience  was  api^lied  to  the  production  and 
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demonstration  of  system  builds.  Ihey  assigned  specific  modules  and  segments  to 
programmers  within  their  gi-oups  and  monitorai  the  development  and  testing  of  the 
program  code.  As  leaders  of  the  build  development  teams  they  provided  technical 
direction  and  guidance  to  team  members  through  demonstration  of  the  build.  Through- 
out the  development  of  this  system  the  Chief  Programmers  were  a key  link  between 
the  project  analysts  and  programmers,  and  the  management  staff. 

3.6.2  MPP  Comparison 

Since  the  Chief  Programmer  function  was  applied  to  only  one  project  there  is  no  basis 
for  comparison  of  its  effects  among  the  other  projects. 

3.6.3  R ec  o m m end  at  io  ns 

The  Chief  Progranimer  position  as  used  on  the  DD-963  project  was  successful  pri- 
marily due  to  these  factors: 

• Size  of  the  project  staff  was  such  that  two  programming  groups  provided 
optimum  coverage  of  program  development  requirements. 

• Programming  groups  were  large  enough  to  encompass  a broad  range  of 
programming  expertise  without  sacrificing  supervisory  control. 

• Individuals  assigned  as  Chief  Programmers  were  able  to  provide  out- 
standing technical  direction  and  control. 

• Department  Manager  provided  administrative  and  program  development 
support  so  that  Chief  Programmers  could  devote  their  full  attention  to 
software  production. 

It  would  be  difficult  to  quantify  all  of  the  factors  which  led  to  the  successful  application 
of  a Chief  Progi’ammer  function  because  many  relate  to  personnel  interactions.  It  is 
possible  that  given  the  proper  mix  of  personnel  a much  larger  project  could  effectively 
use  this  staff  structure.  Certainly,  a smaller  project  could  easily  implement  a Chief 
Programmer  organization  such  as  the  one  DD-063  used. 

It  is  possible  that  with  the  same  project  staff  another  type  of  staff  organization  would 
have  workal  as  well  or  better  than  the  one  used  on  DI)-!)6:!.  To  recommend  using  a 
Chief  Programmer  team  structure  one  must  consider  all  the  factors  mentioned  in 
tliis  discussion,  'iliere  are  no  definitive  guidelines.  It  is  obvious,  however  that  this 
organization  can  be  used  effectively  and  can  enhance  the  technical  direction  and  con- 
trols applied  to  a software  acquisition  project. 

3.7  TUTLl)  DKADKll 

A build  leader  is  a specialized  task  leader  whose  sole  responsibility  is  to  supervise 
[)roduction  of  software  to  support  a build  capability.  .\s  in  most  task  oriented  organi- 
zations the  build  leader  may  have  other  non-related  or  compatible  responsibilities. 
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As  an  example,  Chief  Proi^rammers  were  build  leaders  for  most  of  the  system  builds 
during  one  project.  ITie  build  leader  directed  the  efforts  of  the  tash  group  of  pro- 
grammers and  analysts  assigned  to  a particular  build,  lliis  group  h/^ictioned  as  a 
unit  until  the  build  was  demonstrated.  Build  leaders  were  instnjn:ental  in  timely 
production  of  system  capabilities,  and  in  the  detection  and  correction  of  design  and 
code  errors. 

3.7.1  Development  Chases 

Build  leaders  were  us(xl  only  during  the  Code  and  Checkout,  and  Test  and  Integration 
phases,  lliey  were  usi.xl  with  equal  effectiveness  across  both  of  these  phases.  Per- 
haps the  greatest  benetit  to  he  gained  from  using  build  loaders  is  in  having  software 
development  teams  working  with  well  defined,  limited  objectives.  Work  progress 
can  be  readily  determincxl  and  personnel  can  be  assigned  to  specific  tasks.  The  build 
leader  and  his  group  functiotuxl  as  an  action  team  within  the  project  staff.  Once  the 
build  was  demonstrattxi  these  people  were  reassigned  to  other  tasks.  In  this  way, 
manpower  becomes  a more  flexible  resource  for  use  within  the  project.  This  was 
true  of  tlie  way  build  leaders  were  employed  through  both  development  phases. 

3.7.2  MPP  Comijarisoti 

Data  on  the  use  of  build  leaders  was  only  available  from  the  DD-9r>3  project,  'fhere- 
fore,  no  comparison  of  their  effects  among  the  projects  can  be  made. 

3.7.3  Hecommendations 


If  a management  staif  elects  to  employ  threads  and  builds  to  promote  phased  produc- 
tion and  integration  of  system  capabilities,  then  build  leaders  are  an  appropriate 
teciinique  to  assure  their  etfectivc  use.  Build  le;iders  can  be  used  on  projects  of  all 
sizes.  'Die  project  staff  organization  must  be  structured  so  that  the  build  leader  has 
available  to  him  the  staff  support  he  needs  to  accomplish  his  assigned  tasks.  For 
example  til e DD-0()3  organization  included  configuration  management,  quality  assur- 
ance and  program  librarian  functions  that  were  common  to  alt  task  gi’oups.  ITius, 
the  build  leaders  could  devote  their  full  attention  to  program  development,  'fhe  total 
project  staff  size  was  smalt  enough  to  accommodate  this  staff  organization.  Build 
leaders  and  theii’  groujis  were  alile  to  have  direct  and  frequent  access  to  support  per- 
sonnel during  build  production. 

3.8  BRQ.IKCT  UI  ATKW.S 

llie  review  process  ustxl  In  t’.s'C  on  the  tu’ojects  under  study  included  both  informal 
and  formal  reviews.  Preliminary  Design  Reviews  (I’DIis)  and  Critical  Design  Re- 
views (CDRs)  were  requirerl  by  contract  and  were  formal  reviews.  ■ In  Process 
Reviews  (IPRs)  were  more  informal  and  were  held  throughout  the  development  cycle 
to  provide  the  customer  and  CSC  management,  information  concerning  the  current 
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project  status. 

3.8.1  Development  Pliascs 

For  most  projects,  after  the  CDR  is  conducted  near  the  end  of  Uie  desif^n  phase, 
there  is  very  little  user  inv'olvement  in  the  software  development  process.  Once  the 
detailed  design  is  approved  the  user  must  content  himself  with  patient  anticipation 
while  awaiting  delivery  of  the  final  product.  Ihiis  approach  can  lead  to  development 
of  software  that  does  not  satisfy  user  rec)uirements . Also  the  developer  does  not 
ha\e  a convenient  vehicle  for  advising  the  user  of  f)rogram  development  problems 
which  might  be  resolved  by  modifying  system  design  or  requirements.  Ilie  In  Pro- 
cess Review  (IPR)  is  an  attempt  to  keep  the  user  and/or  program  management  staff 
involved  in  program  development  through  the  period  of  software  production  and  testing. 
IPRs  are  conducted  regularly  and  involve  face-to-face  contact  between  user  and  de- 
veloper personnel . 

Tlie  design  reviews  were  conducted  during  the  design  phase  of  the  project  and  in  most 
cases  after  rhe  CDR  tlie  customer  involvement  with  the  develoi)ment  is  almost  non- 
existent. CSC  attemptexJ  to  keep  the  customer  involved  throughout  the  development 
phases  by  tlie  use  of  regularly  scheduled  in  process  reviews. 

3.8.2  i\lPP  Comparison 

The  design  reviews  for  the  three  projects  were  very  maich  standard,  but  different  in 
number  and  length  because  of  the  size  and  complexity  of  the  projects.  'Die  IPR 
approach,  however,  had  a very  positive  impact  on  the  projects  by  providing  manage- 
ment visibility  and  customer  involvement. 

nie  TRIDF.VT  project  had  very  few  IPRs  in  the  early  phases  of  its  deielopment,  how- 
ever after  \ arious  problems  had  been  encountered  and  the  milestones  were  not  being 
met,  the  customer  became  more  involved.  This  resulted  in  regular  reviews.  These 
reviews  provided  the  customer  with  visibility  into  progress  and  into  the  problem  areas 
and  assisted  in  expedient  resolution  of  problems. 

llie  AEGIS  project  was  conducted  in  a research  and  dei'elopment  environment  with 
changing  reejuirements  and  many  contractors  involved.  IPRs  with  RC.\  and  Navy  per- 
sonnel became  a way  of  life  early  in  the  project  and  were  continutxl  throughout  the 
jiroject.  llie  personnel  interviewed  during  the  research  felt  that  the  IPRs  played  a 
major  role  in  the  success  and  timely  completion  of  the  project,  ilie  data  presented 
at  these  reviews  were  current  and  factual  to  show  the  current  status  of  the  overall 
pi'ojcct. 

Die  DD-9G3  project  held  IPRs  on  a schaluled  basis  at  tiie  customer's  rcc|uest.  'iliis 
was  a time  and  materials  contract  and  the  customer  desirovi  a high  degree  of  visi- 
Inlitv.  The  effect  of  this  tool  was  much  the  same  on  all  projects.  Tlie  reviews 
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established  a good  rapport  with  the  customers  and  instilled  confidence  in  the  product 
being  developed. 

3.8.3  Recommendations 

Project  reviesvs  are  essential  for  visibility  and  control  of  all  software  development 
projects.  ITie  following  recommendations  will  improve  the  conduct  of  these  reviews; 

• llie  review  dates  should  be  scheduleti  as  milestones. 

• The  review  dates  should  be  rescheduled  if  program  development  is  de- 
layed for  any  reason,  llie  work  must  be  complete  in  order  for  the  re- 
view to  be  meaningful  and  not  just  a review  for  the  purpose  of  meeting 
the  niilestone. 

• 'file  iiiuterial  to  be  covered  should  be  sent  to  all  attendees  well  enough 
in  advance  of  the  meeting  for  the  attendees  to  properly  review  the  mate- 
rial. 'lids  pnx'ixiure  will  result  in  more  pertinent  questions  into 
specific  areas  and  ruiuce  the  lime  required  to  conduct  the  review. 

The  use  of  in  process  reviews  is  recommendeti  for  all  projects. 

3.9  PROGRAM.MIXO  i i i U.M'-jIT  ,- 

'lliere  were  three  di-stinct  proararT  min , tediniques  used  during  development  of  soft- 
ware for  these  project-,  t'ollectivel v they  represented  formal  attempts  to  structure 
the  programming  proccc.  . structuring  the  prf)cess  is  important  because  it  provides 
a standurriized  protluct  (ju-ogTam  corJe)  and  intro(.iuces  a procedural  approach  to  what 
has  a tendency  to  be  di  i>i’;,.ini/,<.<l  ictr-  itv  (programming).  'ITie  name  of  the  tech- 
nique or  tlie  paidieular  rul>  s Ahieh  support  it  art.'  less  important  than  the  fact  that  a 
structund  technique  is  usexi  and  cnforcoii.  Ihis  is  the  key  to  success  in  using  any 
programming  tectinique. 

StructurLxl  programming  is  a im  thod  of  writing  programs  with  the  logic  flow  always 
proceeding  from  a single  enti—  through  a single  exit  without  arbitrary  branching.  To 
support  this  methoci.  a system  must  iie  segmented  into  small  programs  that  can  be 
written  with  a limiteil  numl)er  of  logic  structures  to  develop  clettr,  readable  and 
m ;i  i n t a i n ;i  hi  e p r og  r a m s . 

Functional  programming  involves  the  development  of  software  basui  on  functional 
routines  that  are  integrated  to  cre  iti'  systeni  and  subsvstem  capabilities.  'liiis  pro- 
gramming technique  emploves  well  defintxl  coding  conventions  to  produce  routines  in 
standardized  formats  for  use  n ithin  the  system. 

Structured  modular  ina^gramming  is  a modification  of  structured  programming  which 
is  less  restrictive  in  the  area  of  structurexi  constructs.  For  structured  modular  pro- 
gramming the  technical  staff  is  permitted  to  use  "GO  IX)  ' statements  and  may  deviate 
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from  the  binglc  c-ntr\/sin^;lc  exit  requirement  of  structured  proi^ramming.  However, 
the  intent  of  struetunxi  pro^;rammin>f  is  followed  with  respect  to  data  names,  code 
indentinf^  and  prot^ram  lenf^th. 

If . h.  1 Development  I'hases 

'lliese  proi;ramminij:  techniques  are  applied  during  the  program  design  and  coding 
phasi-s,  wiiii  tile  benefits  realized  not  onlv  in  the  coding  phase,  but  in  the  test  and 
maintenance  I'hases  as  well.  For  structurul  programming  the  detail td  design  w'ill 
involve  more  intense  activity  than  for  other  programming  techniques , because  of  the 
level  of  detail  required  for  structured  programming.  However,  this  detail  will  de- 
crease tile  effort  requirui  to  proeiuce  program  code.  .Ml  three  of  these  techniques 
improved  jiroductivity  in  the  test  phase  because  of  the  smaller  (usually  one  program 
per  computer  print  out  page),  less  comple.x  programs.  Also  the  improved  readiliility 
of  the  standardized  coding  structures  will  have  a positive  effect  on  program  mainte- 
nance in  the  operation  and  support  phase. 

3.9.2  MPH  Cominirison 

TTie  comparison  of  these  techniques  over  the  three  projects  reviewed  is  difficult  be- 
cause only  one  project  used  pure  Structund  Pi-ogramming  (SP)  and  the  other  two  used 
a modified  form  of  SP.  The  TOIDENT  project  which  used  the  pure  form  of  ST>  had 
some  problems  during  the  development  of  the  software.  One  of  the  problems  was 
core  usage  under  SP  constraints.  It  was  realized  toward  the  end  of  the  coding  phase 
that  the  core  budget  would  be  overrun.  This  resulted  in  some  reprograniming  to 
optimize  core  usage  while  maintaining  SP  code.  During  interviews  with  the  TOIDENT 
staff,  the  analysts  stated  that  they  felt  the  program  design  required  a longer  period  of 
time  than  that  for  free  form  programs  because  of  the  need  to  define  the  program  struc- 
ture at  a more  detailed  level.  But,  the  coding  phase  was  r<.duced  to  almost  half  the 
normal  time  due  to  the  design  being  developed  at  almost  the  code  1 cvel . 7'he  super- 
visors on  these  projects  thought  the  constraints  of  SP  did  not  effect  the  productivity 
of  the  senior  level  personnel  to  any  degree,  however  a noticeable  adjustment  was  re- 
quired by  the  junior  level  personnel  to  develop  program  logic  in  a structured  manner. 

It  was  also  felt  that  the  quality  of  the  programs  produced  by  the  junior  staff  members 
using  .SP  was  higher  than  would  ordinarily  be  expected. 

The  total  effect  of  structured  programming  on  the  TOIDENT  programs  cannot  be  de- 
termined at  this  time  since  one  of  the  measures  of  effectiveness  is  maintainability 
and  the  product  has  not  reachexi  that  phase  in  its  life  cycle. 

On  the  .AEGIS  project,  as  a test,  several  program.s  were  written  by  (\sr  senior  pro- 
grammers using  the  SP  constraints  and  then  rewritten  using  a structun.d  format  with 
free-form  code.  'Die  results  of  the  test  indicatotl  that  structured  programs  required 
.o'T  to  sr;  more  core.  Since  the  core  allocation  was  a key  factor  on  the  .AEGIS  pro- 
ject a modified  version  of  SP,  functional  programming,  was  selectid  for  this  project. 
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A similar  technique  was  used  on  the  l)l)-903  project  were  structured  modular  pro- 
gramming was  eniployu.1.  'Ilie  modified  versions  of  SP  used  on  the  AKGIS  and  DD-963 
projects  were  successful  in  producing  quality  jirograms  within  the  core  limitations. 
ITie  programs  written  on  these  projects  did  not  adhere  to  SP  constraints,  however, 
both  projects  were  developtxl  under  rigid  c(xiing  conventioas  and  used  a structured 
format  to  ensure  standardization  of  programs.  The  AEGIS  KDM-1  system  has  been 
operational  since  V.)(’4  onboard  the  I'SS  Norton  Sound  with  software  trouble  reports 
(STO)  being  sent  to  CSC  for  corrections. 

3.9.3  Recommendations 


Structured  programming  (Si>)  or  a version  of  structured  programming  would  be  rec- 
ommended for  any  system.  The  more  complex  and  interactive  a system  is,  the 
greater  the  riKiuirement  for  a standard  structure.  lYie  compiler  being  used  will  have 
some  influence  on  any  decision  to  use  pure  SP  or  not.  In  some  cases  a compiler  with- 
out all  of  the  SP  verbs,  or  a preprocessor,  will  generate  inefficient  code  and/or  use 
excessive  core.  However,  whatever  technique  is  used,  there  should  be  specific  cod- 
ing conventions  to  dictate  development  of  struetured  programs  with  straightforward 
logic  flow  which  have  the  characteristics  of  clarity,  readability  and  maintainability. 

3.10  PROGUESSIVi;  TESTING  AND  INTEGU A l'ION 

Progi-essive  testing  and  integration  is  the  process  of  verifying  and  integrating  seg- 
ments of  a system  in  a time  phasal  schalule  as  the  system  is  being  developed.  The 
purpose  is  to  detect  any  errors  as  early  as  possible,  provide  the  customer  with  early 
visibility  and  avoid  the  conventional  process  of  testing  and  integrating  the  total  system 
in  one  process  at  the  end  of  the  development  phase. 

3.10.1  Develo]unent  Phases 

This  technique  is  applird  during  the  code  and  debug  phase  with  the  benefits  being 
realized  through  the  system  integration  phase.  The  immediate  benefit  is  the  visibility 
of  progress  that  is  demonstraide  to  management  and  the  customer,  llie  prime  benefit 
of  progressive  testing  and  integration  is  early  error  detection  and  correction.  Fig- 
ure 3-0  depicts  the  errors  detectui  and  documented  by  trouble  reports  during  a three 
year  span  covering  program  design,  program  dev’elopment,  system  integration  and 
maintenance  for  a large  swstem  dev'elopment  project.  'Die  peaks  on  the  graph  during 
the  first  year  represent  the  errors  detected  during  the  testing  and  integration  on  the 
builds  for  that  year.  'ITiere  were  7.").')  errors  detcctwl  during  the  program  design  and 
program  development  phases.  Without  phased  testing  and  integration  these  errors 
probably  would  not  have  been  detected  until  system  integration  when  the  cost-to- 
correct  is  much  higher.  With  phased  testing  and  integration  the  overall  development 
schedule  is  shorter,  since  the  progressive  testing  and  integration  is  accomplished  in 
parallel  with  the  coding  of  the  next  build. 
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3.10.2  MPP  Comparison 


This  technique  was  used  on  all  three  CSC  projects  reviewed  in  conjunction  with  builds. 
The  THR EADS/Builds  concept  to  identify  functional  components  for  development  and 
test,  an  independent  test  ^oup  to  ensure  all  system  functions  are  tested,  and  a soft- 
ware configuration  management  group  to  control  the  baselined  portion  of  the  system 
arc  all  supported  by  j)rogressive  testing  and  integration.  During  the  integration  of  the 
builds  into  the  baselinal  portion  of  the  system,  regression  testing  is  performed  to  en- 
sure previously  implemented  system  capabilities  were  not  affected.  Tbe  AEGIS  pro- 
ject used  these  techniques  and  the  system  integration  phase  was  reduced  due  to  many 
interface  problems  being  resolved  during  the  integration  of  ljuilds.  Tbis  does  not 
mean  that  there  were  no  problems  encountered  during  system  integration,  but  the  im- 
pact of  problems  detected  during  this  phase  were  greatly  reduced  by  applying  this 
practice  during  the  previous  phases. 

The  DD-9(53  project  was  similar  to  AEGIS  in  that  the  customer  required  a high  degree 
of  visibility.  With  the  use  of  progressive  testing  in  conjunction  with  other  MPP,  CSC 
was  able  to  provide  the  customer  with  current  status  reports  of  the  system  progress 
and  demonstrate  the  system  capabilities  as  they  were  developed. 

The  total  effects  of  progressive  testing  on  the  DD-963  and  TRIDENT  projects  cannot 
be  determined  at  this  time  as  they  have  not  reached  the  system  integration  phase, 
however  based  on  the  results  ol  tlie  .\EGLS  project  and  the  benefits  already  realized 
on  the  DD-9fl3  project,  sucli  as  early  error  detection  and  early  visibility  this  tech- 
nique has  proven  to  be  a very  effective  tool. 

3.10.3  Recommendations 

llie  use  of  this  technique  is  rocommendal  with  any  project  where  communication  be- 
tween system  components  is  reciui''otl.  A system  which  has  a great  amount  of  inter- 
actions, such  as  a command  and  control  system,  will  benefit  the  most  from  pro- 
gressive testing  and  integration.  However,  when  used  in  conjunction  with  the  other 
MPP  discussed  the  cost  of  applying  the  tool  is  minimal  and  the  benefits  derived  make 
it  very  cost  effective. 

3.11  SfPPORT  PROGRAMS 

Support  programs  comprise  all  software  which  is  functionally  exteinal  to  an  opera- 
tional system,  but  executes  in  conjunction  with  that  system  to  provide  diagnostic, 
simulation  and  translation  services.  ITiis  includes  computer  programs  and  sub- 
systems providing  program  compilations,  coredump.s,  timing  data,  message  trap.s, 
selected  memorv  register  dum|is , interface  simul ations  and  interactive  simulations 
and  interacti\  e instniction  manipulation.  In  addition,  on  the  DD-9fi3  project  program 
documentation  was  enhanced  through  the  use  of  an  automated  program  flow  chart 
generator. 
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Support  programs  were  used  on  all  three  projects  studied.  'ITie  AEGIS  project  was 
the  source  for  many  of  the  support  programs  adapted  for  use  on  both  the  'fUIDENT 
and  DD-963  projects.  ITiese  programs  were  implemented  as  former  .\EGIS  staff 
members  recognized  their  utility  under  similar  program  development  conditions  on 
both  'IHIDENT  and  DD-963.  In  particular,  the  DD-9G3  managers  initiated  a compre- 
hensive support  software  package  supplementing  the  AEGIS  programs  with  additional 
support  systems  to  promote  increased  productivity  from  their  limited  staff.  On  both 
the  AEGIS  and  DD-963  projects  support  programs  played  a significant  role  during 
program  development  and  testing. 

3.11.1  Development  Phases 

Support  programs  were  used  on  all  three  projects  studied.  'Dmir  impact  on  the 
development  process  in  each  case  was  heavily  dependent  upon  the  level  of  application, 
lliese  projects  presented  different  management  approaches  toward  their  use. 

This  practice  affected  all  phases  in  the  program  life  cycle  of  the  AEGIS  project.  The 
support  software  was  defined  during  the  design  phase  as  to  the  types  of  software  need- 
ed and  what  phase  in  the  development  cycle  it  would  he  required.  It  was  realized 
early,  that  most  of  the  support  programs  required  to  develop  this  large  weapons  sys- 
tem would  have  to  be  designed  and  developed  by  the  project  staff.  This  decision  was 
influenced  by  the  fact  that  the  computer  to  be  used  was  new  and  very  few  debug  pro- 
grams were  a\'ailable  for  testing  real-time  programs,  also  the  hardware  was  being 
developed  in  parallel  with  the  software  which  meant  that  simulation  programs  would 
be  requirtd  for  software  checkout.  Tliese  factors  and  the  contractually  require>d 
support  software  resulted  in  G3^  of  the  total  system  software  being  support  software. 
(See  Figure  3 -7. ) 

The  suppoi't  software  employed  on  the  TTllDENT  project  consisted  of  debug  routines 
and  simulators  used  during  the  test  phase.  For  the  most  part  these  were  programs 
developed  for  the  AEGIS  program  and  were  resident  in  the  Ih'ogram  Generation 
Center  (i^GC)  libraries.  Both  .AEGIS  and  TRIDENT  shareii  the  DGC  de\  elopment 
facility. 

Tlie  DD-9G3  technical  staff  was  able  to  increase  prcxluctivitv  during  the  code  and 
checkout  phase  through  the  use  of  the  support  programs.  The  technical  staff  was  not 
large  considering  the  size  of  the  system  to  be  developed.  .\lso,  the  customer  had 
specified  a shorter  development  period  than  had  been  originally  proposed.  These 
factors  combined  to  create  a sizeable  workload  on  all  members  of  tlie  technical  staff. 
'Die  department  manager  and  supervisors  statt<l  that  without  the  benefit  of  this  sup- 
port program  package  the  delivery  dates  for  system  and  subsystem  components  could 
not  have  been  met. 

For  the  DD-nG3  project  the  code  and  checkout,  and  test  and  integration  phase  activi- 
ties overlapped.  Through  the  build  concejit,  selected  system  functional  capabilities 
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Figure  3-7.  AEGIS  EDM-1  Program 


were  devolopL'd  and  demonstrated  on  an  incremental  basis.  In  this  way  system  func- 
tions were  integrated  while  being  tested  in  an  orderly  fashion.  The  support  programs 
were  equally  important  during  this  period  when  a great  deal  of  individual  program 
troubleshooting  and  code  modification  was  taking  place.  Tliroughout  this  period  pro- 
grammer producti\dty  remained  high  as  the  project  technical  staff  made  effective  use 
of  the  support  [)rograms. 

'llie  study  of  support  programs'  effectiveness  did  not  extend  into  the  installation  or 
operation  and  maintenance  phases.  Ifie  project  had  not  entered  these  phases  while 
the  software  production  data  research  was  being  conducted. 

3.11.2  MPI*  Comparison 

Support  programs  were  used  very  effectively  on  both  the  AKGIS  and  DD-963  projects. 
For  AKGIS  much  of  the  program  test  software  was  developed  under  the  contract  for 
delivery  to  the  user.  Some  DD-963  support  programs  were  also  delivered  to  the  user. 
In  obth  projects  the  technical  staffs  made  full  use  of  all  available  software  aids  and 
modified  the  programs  when  necessary  to  make  tliem  more  effective.  'Fhis  had  a very 
favorable  impact  on  the  software  develo[)ment  process  for  both  projects.  In  particu- 
lar, the  Di)-963  task  leaders  stated  that  they  would  not  have  been  able  to  meet  sched- 
uled milestones  hat!  it  not  been  for  the  program  debug  tools  available  in  the  support 
software. 

llic  nilDKNT  project  staff  did  not  appear  to  depend  very  heavily  on  the  support  pro- 
grams which  were  available.  As  stated  earlier,  they  used  .\KGIS  support  programs 
that  were  accessible  to  them  through  the  PGC  program  library. 

3.11.3  Recommended  Use 

Support  software  is  required  for  most  large  software  development  projects.  The  type 
and  quantitv  vary  with  the  product  being  devel oped . 'llie  data  collectal  during  this 
research  indicate.s  that  without  the  proper  support  tools  the  successful  completion  of 
these  pmjects  could  not  have  been  obtained  in  the  required  time  frame.  CSC  recom- 
mends evaluation  of  e.xisting  support  programs  that  could  provide,  with  the  least 
amount  of  software  modification,  the  necessary  support  of  system  development  efforts. 

3.12  .SOF-nV/VRK  COXFIGITl A'HON  MAN.XGKMKNT 


Software  configuration  management  (SCM)  is  the  jn’occss  of  controlling  a system  con- 
figuration throughout  its  life  cycle  to  avoid  unauthorized  changes  after  the  baseline 
has  been  established. 
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3.12.1  Development  I’hases 

SCM  is  applied  the  life  cycle  of  a software  system,  from  the  time  the 

design  document  is  approval  and  baselined  through  the  operation  and  maintenance 

phase.  'Hie  benefits  of  this  tool  are  realised  when  a change  or  an  enhancement  to  the 

system  is  rec|uired.  'I'he  documental  or  designed  configuration  must  be  the  same  as  P 

the  operating  configuration  in  order  for  the  maintenance  personnel  to  determine  || 

which  portion  of  the  s>stem  requires  nnxlification.  If  any  part  of  a system  has  been 

"patched"  or  changui  witliout  uixlating  the  documentation,  a modification  or  enhance-  i 

ment  could  effect  the  system  performance  by  incorrect  interface  with  the  patched  q 

portion  of  the  system.  ' ;] 

Tliis  programming  practice  was  applial  to  the  projects  reviewai  and  varied  in  rela-  p 

tion  to  the  amount  of  funds  available  for  configuration  control.  Tlie  AEGIS  project,  \- 

the  largest  on  the  three  studied  had  a configuration  management  section  with  five  | 

personnel  to  control  the  large  volume  of  documentation  and  programs  being  pro-  I 

duced  during  system  development.  During  the  acceptance  testing  phase  this  section  ] 

controlled  multiple  versions  of  the  system  to  c'onform  to  the  various  stages  of  hard- 
ware  configuration. 

Tlie  DD-063  project  had  a one  man  configuration  control  section,  lliis  project  was 

smaller  than  the  .\i;GlS  project  and  control  of  the  baselined  system  was  not  as  diffi-  i' 

cult.  .-Ml  changes  made  to  the  baselined  programs  had  to  be  approvcxl  by  the  chief 

programmer.  .-Ml  changes  to  tiie  system  requiral  approval  of  the  program  Configu-  ij 

ration  Control  Boaixi.  < 


llic  ntntENT  pi-oiect  was  also  a small  project  in  terms  of  personnel  (20-30  technical)  ■ ! 

and  the  configuration  control  was  pnirt  of  the  duties  of  one  person.  The  configuration  ^ i 

control  on  this  project  was  somewhat  lax  because  under  the  conditions  specified  by  ' i 

the  prime  contractor  tlie  system  wasn't  considered  baselined  until  it  was  completed. 

This  version  of  SCM  was  not  as  desirable  as  that  used  other  two  projects. 

3. 12. 2 .MPP  C omixu’ison 

, 1 

The  effect  of  SCM  varial  o\er  the  three  projects  in  proportion  to  the  amount  of  con-  ' 

trol  applied  through  that  function  within  the  project.  'Ihe  lltlDENT  project  where  ,, 

the  smallest  amount  of  resources  was  expendcfl  in  controlling  the  "current"  version 
of  system,  encountcral  some  problems.  The  problems  were  costiv  in  man-hours  1 

and  machine  time  but  were  not  unsurmountable.  ^ 

'Ihe  other  two  projects  which  placet!  more  stringent  control  on  the  baselinod  version 

of  the  system  were  more  effective  in  controlling  changes  to  the  configuration  and  en-  l, 

suring  the  system  performod  in  accordance  with  the  specifications.  i 


In  summary  the  effectiveness  of  any  SCM  function  is  determined  by  the  amount  of  con- 
trol applied  witliin  a j;iven  project. 

3.12.3  ilecommendations 

Idle  use  of  an  SCM  function  is  recommended  for  all  projects,  'ilie  size  of  this  staff 
function  should  be  determintid  by  the  size  of  the  project,  llie  confij^u ration  control 
plan  should  be  developed  during  the  system  design  phase  and  the  plan  should  be  en- 
forced to  ensure  the  integrity  of  the  system  configuration  throut^out  the  life  cycle  of 
the  project. 

3.13  STOl'ClTRED  WALKniROUGHS 

.A  structured  walkthrough  is  the  review  of  a program  design  or  program  code  by  an 
evaluation  committee.  'ITiis  review  is  conducted  prior  to  coding  as  a design  review 
and  prior  to  unit  testing  as  a program  logic  review,  ilie  primary  programmer  pre- 
sents his  design  or  code  to  the  committee  members  who  step  through  the  logic  of  the 
program  to  detect  errors  prior  to  coding  or  testing.  'Phis  technique  is  sometimes 
referred  to  as  peer  group  programming. 

3.13.1  Development  Phases 

'Hie  phase  where  walkthroughs  are  applied  is  tlie  coding  phase.  However,  a walk- 
through of  the  program  design  prior  to  actual  coding  can  be  very  effective  in  detecting 
misinterpretations  of  database  structure,  communication  links  and  the  general 
approach  to  the  solution  of  a problem.  From  this  viewpoint,  the  phases  affected  by 
this  technique  could  be  described  as  the  design  phase  through  the  test  and  integration 
phase,  'niis  techni(|ue  is  similar  to  a mini  critical  design  review  (CDR)  except  that 
it  is  normally  an  internal  process  and  less  formal  than  a CDR. 

'Hus  technique  was  used  on  both  the  TRIDKNT  and  DD-9G3  projects  in  a similar  man- 
ner. UTie  evaluation  committee  averaged  five  people,  and  normally  consisted  of  the 
analysts  whoso  programs  had  interactions  with  the  program  under  review  and  the 
primary  programmer's  immediate  supervisor.  The  design  or  code  was  reproduced 
and  distributed  to  the  committee  members  for  review  prior  to  the  evaluation  date. 

The  reviews  were  internal  to  the  project  and  informal  to  a degree,  to  encourage  con- 
structive criticism  from  the  participants.  Hie  dates  set  for  the  walkthroughs  were 
part  of  the  milestones  for  completion  of  program  design  and  coding  as  scheduled  by 
the  programming  manager. 

3 . 13 . 2 .MPP  Comparison 

'Ilie  effect  of  the  walkthroughs  on  the  projects  reviewed  was  expressed  by  the  person- 
nel on  the  respective  staffs  as  cost  effective  in  both  man-hoia’s  and  machine  time. 

'Ihe  number  of  personnel  involval  in  a walkthrough  ranged  from  three  to  seven  and 


the  time  required,  was  alxjut  four  hours  per  prO|;ram.  'llie  direct  benefit  from  these 
evaluations  was  in  the  test  time  being  reduced  by  50%  to  S0%  because  errors  were 
detected  prior  to  the  test  phase.  Ihe  indirect  benefits  are,  (a)  verification  of  pro- 
gram interfaces  to  alleviate  integration  problems,  (li)  ex(.)anding  knowledge  of  system 
components  over  the  staff  members,  (c)  accelerating  the  software  production  by 
reduction  of  test  time  and  integration  time,  and  (d)  detecting  errors  early  in  develop- 
ment cycle,  thus  reducing  the  cost  of  correction. 

3.13.3  Recommendations 


It  is  recommendtx!  tliat  tliis  technique  lie  appliwi  to  any  project  since  the  costA>enefit 
ratio  appears  to  be  very  favorable.  Ihe  indirect  benefits  derived  from  most  well 
executed  walkthroughs  would  offset  the  cost  of  the  participating  personnel.  This 
techniciue  is  more  beneficial  to  an  interactive  system  where  program  interfaces  are 
critical,  but  could  be  cost  effective  on  stand  alone  programs  as  well,  due  to  the  cost 
of  correcting  errors  that  could  he  detected  early  in  the  development  cycle. 

3.14  IXDF.PKNDKNT  ITST  AND  EVALUATION 

Independent  test  and  evaluation  (1T<^-K)  is  a method  for  conducting  software  verifica- 
tion and  validation.  It  involves  designation  of  a group  or  team  of  technical  personnel 
to  form  an  IT^-E  unit.  'I'hese  personnel,  who  must  be  knowUxlgeable  of  system  re- 
quirements and  design,  are  responsible  for  conducting  all  system  tests  prior  to 
delivering  the  software  to  the  user.  This  group  operates  independent  of  the  system 
development  staff  in  order  to  provide  an  objective  and  unbiased  evaluation  of  the  soft- 
ware. The  primary  objectives  of  this  group  are  to  prove  that  die  software  performs 
as  designed  and  that  the  performance  meets  or  exceeds  all  system  recjuirements . It 
must  establish  that  the  system  can  be  implemented,  error  free,  consistent  with  the 
performance  rerjuircmenls . 

To  accomplish  these  objectives  the  ITflE  group  performs  the  following  functions: 

0 Compare  computer  program  design  to  system  design  specifications 
0 Develop  system  test  plan.'^  and  scenarios 

0 Evaluate  software  performance  against  system  ijerformance  requirements 
3.14.1  Development  Phases 

To  develop  an  effective  lest  ])lan  the  preparation  should  begin  during  the  design  phase. 
The  test  and  integration  scripts  for  the  .\EGIS  and  DD-S)G3  projects  were  designed  to 
be  executed  in  conjunction  with  build  demonstratioas . The  unit  testing  was  done  by 
the  individual  programmers  prior  to  the  ilemonstrations  and  the  formal  testing  was 
performotl  by  test  groups  using  the  test  scripts  to  ensure  that  the  functions  designed 
into  a S’pecific  build  satisfied  the  requirements  for  that  portion  of  the  system. 


Using  the  build  concept  and  iniegrating  each  build  as  it  was  developed  and  tested  into 
the  l>aselined  system  diu’ing  the  programming  phase  reduced  the  time  required  for 
the  final  system  test  phase. 

3.11.2  MPU  Comi)arison 

'llie  .Al'.tlLS  project  had  a Test  and  Uvaluation  Department  which  reported  to  the  pro- 
ject manager.  This  group  generated  the  test  plans  and  procedures  for  all  tests,  con- 
ductai  and  evaluated  the  tests  and  retested  any  programs  which  required  corrective 
action  from  a previous  test.  .All  test  plans  and  procedures  were  approved  by  the 
customer  prior  to  the  actual  test  and  the  customer  reviewed  test  results  after  the 
test. 

Ihe  l)D-‘Jb3  project  staff  had  a test  group  which  worked  jointly  with  the  Navw  in  w-rit- 
ing  the  test  scenarios  from  the  threads  data  base,  lliis  group  reported  to  the  DD-963 
department  manager.  The  test  scripts  were  the  scenarios  used  during  a build  demon- 
stration to  ascertain  if  all  functions  in  the  build  perform  as  specified  in  the  require- 
ments document.  'Ihe  results  of  the  test  were  evaluated  and  software  trouble  reports 
(S'n?)  were  written  on  any  problem  detected  during  the  test.  'I'he  Sill  was  tracked 
by  this  group  until  the  error  was  corrected.  'Hie  test  script  was  executed  again  to 
confirn\  the  correction  and  ensure  the  corrective  code  did  not  effect  another  part  of 
the  build.  All  tests  were  made  with  Navy  personnel  present  and  the  programs  base- 
lined after  the  demonstration. 

'Hte  formal  testing  of  the  'IIUDKNT  project  will  be  the  responsibility  of  IBAI  and  thus 
there  were  no  data  available  during  our  investigation  of  this  project. 

'Ihose  invoh  ed  in  directing  proga’am  development  for  both  AKGLS  and  DD-963  stated 
that  getting  the  user  involv  ed  in  the  testing  process  proved  to  be  very  successful 
from  two  standpoints.  First,  this  provided  those  developing  the  software  with  a first 
hand  look  at  how  the  user  intends  to  use  the  capabilities.  Second,  the  user  becomes 
more  familar  witli  the  praluct  and  can  provide  comments  to  clarify  any  operational 
inconsistencies. 

3.11.3  llecominendations 


ITiiK  groups  should  be  used  on  all  software  development  projects  when  there  are  ade- 
quate resources  to  support  them,  'nic  costs  associated  with  employing  an  IT&E  group 
can  be  as  high  as  10  ' of  the  total  program  costs.  Independent  test  and  evaluation  is 
a valuable  MT’P  and  should  be  employed  recognizing  the  following  conditions: 

• IlAvK  personnel  disassociated  from  software  development  staff 

• Test  plans  developed  early  using  requirements  documents  and  then 
basel  inal 

• User  involved  directly  in  testing  to  the  extent  possible 
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3.15  VERIFICAllON  PROCEDURKS 


Formal  verification  procaiures  were  used  only  on  the  DD-963  project.  Computer 
program  design  and  code  were  veriiicd  on  all  of  the  projects  using  MPP  such  as 
THREADS,  Builds,  I'DR/CDR,  etc.  However,  it  was  on! y on  the  DD-9G3  project 
that  verification  procedures  were  specified  through  documentea  project  operations 
directives. 

There  were  two  verification  processes  used  on  the  DD-963  project.  In  order  to  veri- 
fy that  the  program  design  met  the  CPPS  requirement  a top  down  verification  pro- 
cess was  used.  ITien  the  program  code  was  verified  against  the  CPPS  using  bottom 
up  verification  procedures. 

Top  down  verification  was  implemented  through  a multi-step  process  using  threads 
and  builds,  lliat  process  proceeded  as  follows: 

• Prior  to  each  build  demonstration  each  primary  programmer  reviewed 
the  iirogi'ammed  threads  for  wMch  he  was  responsible  to  verify  their 
correctness  against  the  CPPS. 

• build  demonstration  scenario  was  developed  which  assured  execution 
of  every  thread  within  that  particular  build. 

• llie  build  demonstration  was  conducted  to  confirm  specific  functional 
capabilities  associated  with  that  build. 

• Monitor  runs  (trial  executions)  were  conducted  for  selected  programs 
or  modules  to  confirm  questionable  thread  execution  which  occurred 
during  the  build  demonstration. 

• Post  build  testing  was  cmplovLd  to  insure  that  the  final  post  build  system 
configuration  was  functionally  acceptable.  It  consisted  of  the  following: 

Exhaustive  testing  to  execute  functions  over  the  entire  range 
of  values  for  input  data. 

Negative  testing  to  evaluate  system  responses  to  erroneous 
input  data. 

Degradation  testing  to  assess  system  operation  under  sub- 
system fault  conditions. 

Bottom  up  verification  was  achieved  primarily  through  the  use  of  support  programs 
as  program  code  was  developed.  'Hie  elements  of  this  process  were: 

• Compare  automated  flowchart  generator  statements  with  the  program  or 
segment  code  to  assure  a correspondence. 

a Create  program  or  segment  flowcharts  using  the  automated  flowchart 
generator. 

« Verifv  that  the  proi:ram  or  segment  flowcharts  are  consistent  with  the 
CPDS. 
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3.15.1  Development  Phases 


ITiese  verification  procedures  were  first  implemented  during  the  code  and  checkout 
phase  when  threads  were  verified  against  the  performance  specifications  in  prepara- 
tion for  build  demonstrations.  ITiey  continued  to  be  used  through  the  test  and  inte- 
gration phase  as  bottom  up  verification  was  conducted  to  confirm  that  the  program 
code  paralleled  the  design  specifications.  The  net  result  of  these  activities  was 
reflected  in  the  following; 

• Karly  definition  of  subsystem  and  module  interfaces 

• Comprehensive  design  and  code  analysis 

• Error  detection  enhanced  throughout  entire  period  of  development 

• Subsystem  functions  proved  through  exhaustive  testing 

• Viability  of  system  and  subsystem  threads  confirmed 

3.15.2  MPP  Comparison 

Ihis  programming  practice  was  used  exclusive!}'  on  the  DD-963  project,  and  there- 
fore, no  comparison  can  be  made  of  its  effects  on  other  projects. 

3.15.3  Recommendations 

The  programming  directive  that  specified  the  verification  procedures  to  be  used  by 
the  DD-963  technical  staff  was  a clear  definition  of  the  methods  to  be  used  in  verify- 
ing the  software.  It  was  a document  that  all  members  of  the  technical  staff  could  use 
as  a guide  during  program  production  and  testing.  Perhaps  its  greatest  benefit  is 
that  it  offered  a coordinated  verification  plan  using  the  tools  and  techniques  available 
to  the  staff.  For  this  reason  this  practice  is  to  be  recommended  as  an  example  of  a 
viable  plan  for  software  verification.  This  type  of  directive  could  be  applied  to  any 
software  development  project. 
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SECTION  4 - COMPARISON  PROCESS 


4.1  iN'moDuc'noN 

Comparison  of  CSC  pro^'amming  practices  and  sti-uctured  progi'amming  technolog>' 
techmques  is  an  essential  prerequisite  for  tlie  definition  of  MPP  standards,  lliis 
comparison  of  practices  and  techniques  as  descriljod  herein  is  based  upon  analysis 
of  prograximiing  practices  used  by  CSC  on  the  three  projects  researched  and  upon 
descriptions  of  SPT  techniques  found  in  the  IBM  Structured  Programming  Series.  A 
methodology  for  comparing  the  practices  and  techniques  has  been  devised,  niis 
methodologv'  is  founded  upon  the  percent  of  time  specific  error  types  exist  and  the 
probability  of  detection  or  reduction  of  that  error  type  by  the  practices  and  techniques 
within  each  life  cycle  phase.  Using  this  data  and  statistical  analysis  techniques, 
each  practice  or  technique  can  be  assigned  an  effectiveness  value  for  each  life  cycle 
phase,  'ihese  values  will  be  used  later  to  support  the  selection  of  MPP  that  will  be 
proposed  as  standards  for  Air  Force  software  engineering  projects. 

Computer  program  life  cycle  phases  provide  the  framework  within  which  comparison 
of  practices  and  techniques  is  conducted.  Life  cycle  phase  definitions  contained  in 
Air  Force  system  acquisition  management  directives  (AFM  SOO  series)  are  used  for 
the  comparisoi}  process,  lliose  phases  are;  Analysis,  Design,  ('ode  and  Checkout, 
Test  and  Integration,  Installation,  and  Operating  and  Support,  'iliese  phase  defini- 
tions differ  from  the  definitions  contained  in  the  Structured  Programming  Series. 
Correlation  between  the  two  life  cycle  phase  definitions  is  shown  in  Figure  4-1 . ITie 
AFM  bOO  series  phase  definitions  will  be  used  throughout  this  section. 

ITic  MPP  research  and  data  collection  did  not  extend  into  tlie  Installation  and  Opera- 
ting and  Support  phases.  ITae  reason  for  this  is  that  the  CSC  project  data  did  not 
cover  tiiose  phases,  lliereforc,  the  discussions  and  analysis  of  practices  and  tech- 
niques jicrformance  does  not  include  tiiose  phases.  For  two  of  the  projects,  CSC 
contractual  support  did  not  extend  into  those  pliases  in  the  same  form  as  that  re- 
searched. 41ie  third  project  had  not  reached  the  Installation  phase  during  the  period 
of  the  research. 

4.2  COMPARISON  .MK'niODOLOGY 

CSC's  comparison  methodology  is  a stciiwise  process  firoceoding  from  identification 
of  programming  practices  and  techniques  to  calculation  of  their  individual  effective- 
ness values,  llie  fundamental  premise  upon  which  this  methodolog\-  rests  is  that 
the  timely  prcxluction  of  error  free  computer  programs  is  the  goal  of  software  ac- 
quisition projects.  Computer  jirogram  errors  are  definu!  to  include  not  only  pro- 
cessing faults,  liut  also  failure  to  satisfy  system  performance  re(|uircments.  'lliis 
broad  definition  is  meant  to  be  inclusiv  e of  all  aspects  of  computer  program  per- 
formance. 
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In  developing-  the  comparison  methodologj'  four  major  assumptions  were  made.  First, 
practices  and  techniques  that  promote  the  detection  or  reduction  of  computer  program 
errors  can  be  ranked  aceoixling  to  their  effectiveness  with  regard  to  that  capability. 
Second,  each  practice  or  technique  included  in  this  study  has  been  ustd  to  further  the 
timely  production  of  error  free  software.  'Iliird,  it  is  possible  to  define  an  inclusive 
set  of  software  error  types  that  can  be  readily  identified  during  any  software  develop- 
ment. Fourth,  data  can  be  collected  that  w'ill  suppoz-t  definition  of  pi-obability  of 
error  type  existence  and  probability  of  error  type  detection  or  reduction. 

In  the  following  discussions,  values  that  are  used  in  the  matrices  and  in  the  calcula- 
tions are  not  universally  supported  by  project  research  data.  'Ihere  arc  several 
reasons  for  this,  'llie  Software  Production  Data  pi'oject  involved  research  of  only 
three  CSC  software  acquisition  projects  and  the  IBM  Stz-uctured  Ih-ogramming  Series. 
'Phis  meant  that  available  data  points  to  support  conclusions  were  limited  or  non- 
existent. Much  of  the  available  resource  material  was  not  designed  to  support  re- 
search of  this  type.  In  many  cases  the  materia!  could  not  be  substantiated  or  expanded 
because  project  personnel  had  transferred  or  left  the  Company.  To  counter  these 
conditions  the  research  staff  used  statistical  techniques  and  CSC  management  staff 
surveys.  'Ihrough  these  aids  a consistent  and  reasonabl e data  analysis  was  completed, 
although  CSC  admits  that  this  data  is  by  no  means  complete,  it  is  felt  that  enough  of  a 
data  base  exists  to  make  a first-cut  rough  estimate — at  least  to  the  extent  of  trying  to 
provm  the  usefulness  of  the  methodolog}\ 


CSC's  comparison  methodology  is  presented  as  a viable  tool  for  the  evaluation  and 
ranking  of  all  known  MPP.  It  establishes  an  unbiasal  analysis  process  that  may  be 
applied  to  all  types  of  software  acquisition  projects.  While  the  data  available  for 
analysis  of  CSC  software  production  data  were  limited,  the  conclusions  drawn  are 
believed  to  be  realistic.  Given  a more  comprehensive  and  extensive  data  resource, 
the  validity  of  this  methodology  could  be  more  fully  substantiated.  It  is  important 
that  in  the  following  discussions  the  comparison  methodology  itself  be  given  considera- 
tion ecjual  to  that  of  the  comparison  results.  'Ihe  methodolog-\'  is  comprised  of  these 
activities : 

» Identification  of  practice  or  technique  ph;ise  applicability 
<»  Definition  of  software  error  types 
»»  Kstabl ishment  of  percentages  of  error  type  existence 
« Establishment  of  error  type  detection  or  reduction  probabil  ity 
« Calculation  of  practice  or  technique  effectiveness  values 


•1 . 11 . 1 Phase  .\ppl  icabil  ity 

'Hie  C.SC  comparison  methodologv-  is  a life  cycle,  phase-deizendent  ])rocess.  Each 
life  cycle  phase  will  be  a basis  for  conducting  the  complete  comparison  process. 

Onlv  programming  practices  and  techniques  that  are  actively  applied  in  a given  phase 
are  compai'cd  in  that  phase,  'nierefore,  it  is  essential  that  the  phase  applicability 
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of  each  practice  and  techtiique  be  established. 
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'fhe  applicable  phases  for  CSC  programming  practices  comparison  have  been  es- 
tablished through  the  research  conducted  by  tlie  MPP  Project  staff.  For  the  IBM  SPT 
techniques  it  was  necessary  to  refer  to  the  Structured  Programming  Series  for  this 
information.  In  Volume  XIII  of  that  Series  the  SI’T  techniques,  related  SPT  tech- 
niques and  non-SPT  techniques  witli  future  potential  are  identified,  'lliey  are  listed 
on  graphs  depicting  their  life  cycle  phase  applicability.  Ifiese  graphs  were  the 
source  for  their  phase  applicability  definition  in  this  report,  'fhe  non-SPT  tech- 
niques with  future  potential  are  included  because  they  are  discussed  in  the  Series. 

In  addition,  one  SP'f  teehnif|i.ie  was  not  listed  in  Volume  Xin,  but  was  discussed  in  the 
te.xt  of  other  Serit;s  volumes,  that  technique.  Program  Review,  is  included  with  the 
other  techniques  in  this  report  and  an  applicable  phase  was  established  through  ref- 
erence to  the  text  discussion. 

Phase  applicability  of  the  practices  and  techniques  is  graphically  portrayed  in 
Figure  4-2.  ib-actices  and  techniques  are  merged  into  one  listing  under  the  column 
heading  of  Programming  Practices.  'Iheir  applicability  through  the  life  cycle  phases 
is  shown  on  the  bar  graph,  'fhe  programming  practices  have  been  grouped  into  tw'O 
subsets;  technical  and  management.  'Phis  differentiation  will  be  significant  later  in 
the  comparison  f)rocess,  as  will  the  ordering  of  the  practices  and  techniques  in  the 
list.  'Hie  Uesigni  phase  of  the  computer  program  life  cycle  has  been  divided  into 
Functional  and  Detailed  subphases,  'iliis  has  been  done  to  better  define  the  phase 
applicability  of  some  programming  practices. 

4.2.2  Error  lVj)es 

llie  universe  of  software  errors  may  be  grouped  into  discrete  subsets  based  on 
characteristics  unic)ue  to  the  cri’ors  in  each  subset.  'ITiese  subsets,  or  error  types, 
have  been  variously  defined  by  different  industry  studies.  Tlie  error  types  established 
for  this  study  are  tuiupttil  from  those  defined  by  'IllW  Systems  Group  for  HADC  and 
publishal  in  a Software  Reliability  Study  report,  R.\DC-'ni-74-2.'50,  dated  October, 
1974.  Hie  type  definitions  used  during  the  Software  Production  Data  (MPPl  research 
are  generally  more  inclusi\e  than  those  in  the  'niW  report,  lliis  has  been  done  so 
that  the  number  of  error  tyims  is  more  manageable.  An  additional  error  type  was 
also  introduced  in  this  study,  'nming  errors  were  isolated  as  a subset  because  of 
their  critical  significance  to  real-time  software  development  projects  such  as  those 
studied  during  tins  research  task. 

'iTie  error  type  definitions  which  folloc'.  ha\e  been  used  throughout  this  study  to  estab- 
lish MPP  effectiveness  calut^s: 

• Computational  - inac'curate  mathematical  operations.  Errors  in  this 
category  occur  when  a w rong  equation  or  convention  is  used  causing 
erroneous  mathematical  out]uit  wittiin  a routine  oi-  lu'ogram. 
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Software  Developnent  Cycle 

Figure  4-2.  Programming  Practices  A[>i)licability 


;?  • Logic  - faulty  routine  or  program  decision  flow.  Included  in  this  category 

j are  errors  caused  by  incorrect  condition  tests,  status  checks,  model 

^ definition  and  data  storage  references. 

• Input/Output  - improper  data  reception  or  transmission.  These  errors 

’ related  to  data  format,  field  size,  position,  completeness  and  control. 

• Data  Handling  - incorrect  data  manipulation  within  a routine  or  program. 
Errors  made  in  reading,  writing,  moving,  storing  and  modifying  data 

' items  internal  to  a routine  or  program. 

• Operating  or  Utility  Systems  - all  errors  attributed  to  these  systems. 

: • • Configuration  - incompatibility  of  applications  and  operating  system 

[.  software.  ITiis  category  includes  the  catastrophic  errors  and  unexplain- 

I ^ able  program  halts  caused  when  applications  programs  violate  operating 

system  conventions. 

• Software  Interface  - incompatible  data  flow  across  routine  or  program 

, ■ ' boundaries,  niese  errors  result  from  inconsistent  data  transmission 

' protocols  between  software  components. 

• Hardware  Interface  - incompatible  hardware/software  interaction.  Any 
error  that  may  be  attributed  to  data  flow  between  hardware  and  software. 

• User  Interface  - incompatible  dependent  manual /automated  functions. 
Errors  at  the  user  interface  including  the  machine  operator,  manual  or 
data  caixl  inputs,  tape  inputs,  etc. 

• Data  Interface  - incompatibility  between  data  base  structure  and  using 
routine  or  program. 

• Timing  - failure  to  maintain  normal  program  execution  or  system  opera- 
tion due  to  data  manipulation  timing.  Errors  caused  by  the  inconsistent 
operation  of  time  sensitive  processing  components. 

• Requirements  Compliance  - failure  to  provide  a capability  specified  in 
the  requirements  document. 

For  the  purpose  of  comjjleting  this  research  it  was  assumed  that  all  software  errors 
could  be  characterized  as  one  of  the  above  types.  Certainly,  there  are  errors  that 
do  fit  more  than  one  of  the  above  type  defimtions . In  the  broadest  sense  it  can  be 
hj-pothesized  that  the  error  types  defined  above  will  include  all  known  software  errors. 
ITiese  type  definitions  are  sufficient  to  support  implementation  of  this  comparison 
methodology.  It  is  possible  that  more  extensive  research  would  result  in  refinement 
or  addition  to  these  types. 

4.2.3  Error  Existence 

Throughout  the  computer  program  lifi  cycle,  as  any  software  system  is  being  devel- 
oped, programming  errors  are  generated  and  correctal  by  the  development  staff. 
These  errors  occur  and  are  removed  at  rates  that  are  dependent  upon  factors  such 
as  complexity  and  size  of  the  software  system  being  developed,  and  size,  skill  and 
organization  of  the  development  staff.  Tlie  relationships  among  all  factors  (even  their 
positive  identification)  and  the  error  rates  have  not  been  clearly  defined.  ITiere 
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simply  is  not  enou;j;h  empirical  data  to  support  definition  of  these  relationships. 

Information  gathered  during  research  of  the  three  CSC  projects  was  not  definitive 
enough  to  substantiate  any  hypothesis  relative  to  the  rate  of  error  occurrence  or 
removal.  However,  through  project  staff  interviews  and  surveys,  the  .MPP  research 
team  gathered  information  which  lead  to  definition  of  a prototype,  error  existence 
matrix.  'Ihis  matrLx,  shown  as  Pig'ure  4-3,  incorporates  error  type  occurrence  and 
removal  rates  in  one  set  of  values  representing  the  percentage  of  all  error  types 
existing  across  the  life  cycle  phases,  llie  values  reflect  percentage  of  time  that  a 
given  error  tyiie  will  be  generated  in  a specific  phase  and  remain  in  the  software 
through  subset|uent  phases.  MatrLx  values  for  each  phase  (matrix  column)  will  sum 
to  1.00  indicating  that  the  error  percentages  are  inclusive  of  all  errors  for  each 
pi.^se.  The  values  in  this  matrLx  are  representative  of  error  t\pe  existence  from 
the  three  CSC  projects  studied, 

4,2.4  Prror  Detection  or  Reduction 

Programming  practices,  in  general,  function  to  improve  de\elopment  staff  perfor- 
mance, assure  adherence  to  the  production  schedule,  and  insure  maintenance  of 
programmul  production  costs.  Software  error  occurrence  and  removal  represent  a 
common  denominator  for  practice  performance  measurement  among  the  three  prac- 
tice functions.  Performance,  schedule,  and  cost  are  impacted  by  the  presence  or 
absence  of  software  errors  througliout  the  devel opment  process,  llierefore,  prac- 
tice effectiveness  in  all  functional  areas  is  related  to  the  detection  or  reduction  of 
software  errors,  'ilie  operative  relationship  between  practice  performance  and 
error  detection  or  reduction  has  not  been  explicitly  determined.  However,  a 
realistic  characterization  of  the  relutionshij)  can  be  hypothesized,  lliis  may  be  done 
by  establishing  probability  values  for  the  detection  or  raluction  of  error  types  by 
each  programming  practice  in  each  of  the  life  cvcle  phases. 

A matrLx  containing  probability  values  for  the  detection  or  rexiuction  of  error  t\-pes 
by  all  programming  practices  is  shown  as  Figure  4-4.  'Ilie  \alues  in  this  matrLx 
have  lieen  develo|)od  for  the  Code  and  Checkout  phase  as  is  indicatal  at  the  top  of  the 
fignire.  Hccause  of  the  life  cycle  phase  depen<lency  of  programming  practice  per- 
fonnance,  values  such  as  these  must  be  defined  IV  r each  phase,  'nierefore,  a 
unique  detection  or  reduction  probability  matrLx  will  be  necessary  for  each  phase. 
This  report  will  deal  with  only  the  Code  and  Checkout  phase  in  presenting  research 
findings  and  developing  the  comparison  methodology,  'lliere  are  two  reasons  for 
tills  decision.  First,  the  code  and  checkout  [ihasc  actii’ities  imolve  use  of  more 
(irogramming  practices  than  any  other  phase.  This  means  values  for  most  program- 
ming ))ractices  appear  in  the  matrices.  .Second,  the  resource  material  for  this 
phase  was  more  comprehensive  than  for  any  of  the  other  phases.  It  should  be  pointed 
out  that  the  \alues  in  the  matrLx  were  i|ualitativel\  extractixl  from  the  CSC  projects 
reviewed , 
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Using  Figure  4-4  the  probability  of  error  detection  or  reduction  by  any  programming 
practice  in  the  Code  and  Checkout  phase  can  be  determined.  For  instance,  Top  Down 
Design  has  a 59r  pz’obability  of  reducing  computational  errors  in  this  phase. 

4.2.5  Effectiveness  Values 


Programming  practice  effectiveness  values  may  be  defined  for  each  practice  relative 
to  all  error  types  within  each  applicable  phase.  ITzus,  effectiveness  values  are  a 
function  of  the  percentage  of  time  error  types  exist  and  the  probability  of  practice 
detection  or  raiuction  of  tiiose  error  types.  The  purpose  of  mathematical  manipula- 
tion of  the  two  values  is  to  define  a figure  of  merit  or  effectiveness  value  that  can  be 
derived  for  all  programming  practices . Using  this  value,  programming  practices 
can  be  compared  quantitatively  within  or  across  all  life  cycle  phases.  Naturally,  the 
effectiveness  values  are  only  as  accurate  as  the  probability  values.  'The  probability 
values  presented  in  Figures  4-li  and  4-4  of  this  report  are  not  based  on  quantitative 
measurement.  'Iliey  represent  a consensus  of  e.xperienced  assessment  by  CSC  staff 
members.  However,  given  adequate  source  material  and  research  tools  the  actual 
probability  values  could  be  derived.  In  any  case,  CSC's  comparison  methodology  does 
offer  a viable  approach  to  programming  practice  evaluation  and  comparison.  ITie 
values  shown  in  Fii^vros  !~:i  and  4-4  are  valid  for  the  three  CSC  projects  studied, 
'nierefore,  they  support  the  comparison  process  to  be  described  in  the  following  dis- 
cussions. 

Development  of  effectiveness  values  is  a two  part  process.  First,  the  practice 
effectiveness  values  are  calculated  for  each  life  cycle  phase.  Next,  these  practice 
effectiveness  values  are  usai  to  develop  phase  effectiveness  values  for  each  practice 
across  all  life  cycle  phases.  In  the  following  subsections  each  of  these  activities  is 
described  using  probability  values  for  the  Code  and  Checkout  phase. 

4. 2. .5.1  I’ractice  Effectiveness  Values 


Practice  effectiveness  values  are  calculated  using  the  percentage  of  error  types  exist- 
ing and  error  typc'  detection  or  reduction  iirobabilities.  Ihe  calculation  invoh-es 
selecting  a programming  practice,  error  type  and  life  cycle  phase.  Tlien,  the  per- 
centage of  error  type  existence  is  multipliul  by  the  probability  of  error  type  detection 
or  reduction.  ITiat  product  is  multiplied  by  iOO  to  normalize  the  value.  As  an  exam- 
ple, using  the  Top  Down  Design  programming  practice  and  the  Computational  error 
type  in  the  Cf>de  and  Checkout  phase  the  effectiveness  value  calculatal  would  be; 


Existence 

> 

Percentage 

Detection, 'Kc<luction 

: X 

Probabil  ity 

100 

Effectiveness 

\’alue 

0.11  X 

0.0:')  X 

100 

0.  55 
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Completing  the  calculations  for  all  practices  and  error  types  in  the  Code  and  Check- 
out phase  will  produce  the  matrix  shown  as  Figure  4-5.  i’ractice  effectiveness 
against  any  error  type  may  be  found  by  reference  to  the  matrix,  'ihe  values  are 
significant  only  through  comparison  with  other  practice  effectiveness  values.  The 
Total  Effectiveness  column  in  Figure  4-5  represents  the  sum  of  values  for  each  row 
or  programming  practice.  These  totals  will  be  used  later  in  the  comparison  pro- 
cess. Completion  of  these  calculations  for  all  life  cycle  phases  will  result  in  a 
series  of  matrices  that  may  be  used  in  a life  cycle  comparison  process. 


4. 2. 5. 2 Phase  Effectiveness  Values 

Phase  effectiveness  values  are  designed  to  provide  quantitative,  comprehensiv'e 
values  that  represent  a measure  of  programming  practice  performance  against  in- 
dividual error  types.  These  values  will  serve  as  inputs  to  the  process  of  developing 
phase  effectiveness  values.  An  algorithm  has  been  defined  that  uses  practice  effec- 
tiveness values  to  produce  a value  that  defines  practice  utility  within  a phase.  These 
phase  effectiveness  values  may  be  used  to  compare  practices  in  an  unbiased  fashion. 

In  developing  the  phase  effectiveness  algorithm  consideration  was  given  to  certain 
aspects  of  the  practice  comparison  process  and  some  assumptions  were  made.  First, 
the  probability  of  error  existence  matrix  was  considered  to  have  established  the 
relative  distribution  of  error  types  across  the  life  cycle.  It  represents  an  indirect 
statement  about  the  volume  of  errors  by  type  that  will  exist  in  any  phase.  Tliis  is 
an  important  factor  when  evaluating  practice  performance.  Ne.xt,  the  phase  effec- 
tiveness values,  when  calculated,  must  account  for  error  types  that  practices  are 
most  and  least  effective  against.  In  addition,  the  relati\’e  volume  of  those  particular 
errors  on  a phase  by  phase  basis  is  important.  Finally,  the  goal  for  development  of 
the  algorithm  was  to  create  a procedure  that  would  promote  selection  of  an  optimum 
mix  of  }H’actices  for  use  across  all  life  cycle  phases.  It  was  assumed  that  all  error 
types  are  of  equal  significance  to  the  software  developmetu  process.  That  is,  a 
logic  error  is  just  as  significant  as  a timing  error.  Error  types  were  not  given 
weighting  factors.  It  was  also  assumed  that  practice  effectiveness  against  the  most 
and  least  frecjiiently  encountered  errors  in  a phase  does  reflect  the  quality  of  that 
practice. 

Ihe  phase  effectiveness  algorithm  uses  the  following  input  data;  practice  effective- 
ness value  for  the  error  type  with  the  greatest  percentage  of  existence  (labeled  'a'), 
practice  effectiveness  value  for  the  error  type  with  the  least  probability  of  existence 
(labelet!  'b'),  practice  total  effectiveness  value  (labeled  'c"l,  and  the  number  of  error 
types  (labeled  'd').  'Ihe  algorithm  is  expressed  as  follows: 


P 


' si 


i 

i; 


(a  “ bj 
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Figure  4-5.  Practice  Effectiveness 


To  demonstrate  its  use,  given  the  Top  Down  Design  programming  practice  and  the 
Code  and  Checkout  phase,  the  following  data  would  be  input: 


Practice  Effectiveness  Values: 

Logic  Errors  (encountered  most  often)  - 2.20 
Hequirements  Compliance  (encountered  least  often)  - 1.20 
Total  Effectiveness  Value  - 14.78 
Number  of  Error  Types  - 12 

llie  algorithm  would  then  be  executed  as  follows  to  calculate  the  phase  effectiveness 
value: 


(2,20  + 1.20) 


14.78 


Tliis  calculation  sequence  is  repeated  for  each  programming  practice  in  the  Code  and 
Checkout  phase.  'JTie  resulting  values  are  shown  in  Figure  4-6.  Only  the  Code  and 
Ciieckout  column  is  completed  because  input  data  for  the  other  phases  has  not  been 
defined,  lliese  values  establish  the  relative  merit  of  the  programming  practices 
within  the  Code  and  Checkout  phase. 

4, 2. 5. 3 Effectiveness  Value  Benefits 

Certain  benefits  are  realized  through  the  use  of  effectiveness  values  in  the  program- 
ming practice  comparison  process.  The  most  obvious  benefit  is  the  capability  to 
develop  discrete  measures  of  practice  performance  through  a consistent  and  com- 
prehensive series  of  operations,  llie  practice  effectiveness  value  matrices  permit 
the  determination  of  programming  practice  voids,  specific  instances  where  no 
practice  is  effective  against  a particular  error  type.  Ihis  situation  would  indicate 
an  endemic  deficiency  within  the  progi'am  development  [jrocess.  The  phase  effec- 
tiveness value  matrLx  provides  the  vehicle  for  optimization  of  error  detection  through 
identification  of  the  minimum  number  of  programming  practices  that  generate  a maxi- 
mum set  of  values.  Analysis  of  the  phase  and  practice  effectiveness  v'alues  could 
provide  a global  measure  of  anticipated  product  reliability.  By  adding  cost-to- 
dctect  and  cost-to-fLx  data  to  these  matrices  it  would  be  possible  to  develop  minimum 
cost  approaches  to  program  development. 
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SECTION  5 - RECOMMENDED  MPP 


5.1  INTOODUCTION 

Selection  of  the  program tning  practices  to  be  recommended  as  standards  for  Air 
Force  software  engineering  projects  is  the  culmination  of  this  MPP  comparison  pro- 
cess. 'Phis  process,  vvliich  began  with  research  of  three  CSC  software  engineering 
projects,  has  lead  to  the  identification  and  description  of  all  programming  practices 
that  were  used.  Iliese  programming  practices  include  both  management  and  techni- 
cal disciplines  apidied  in  all  phases  of  computer  program  development.  A compari- 
son methodology  has  been  defined  that  permits  unbiased  and  definitive  comparison 
of  the  programming  practices.  The  comparison  methodology  has  been  employed  to 
establish  a basis  for  analyses  of  CSC  programming  practices  and  IBM  SPT  techniques. 
Application  of  the  comparison  methodology  as  described  in  this  report  has  produced 
a quantitative  assessment  of  each  practice  and  technique  in  the  form  of  phase  effec- 
tiveness values  for  the  Code  and  Checkout  phase.  Now,  selection  procedures 

will  be  used  to  identify  those  practices  and  techniques  that  will  be  recommended  as 
Air  Force  standards. 

On  the  surface  it  might  appear  that  the  CSC  comparison  methodology  has  reduced 
MID’ definition  to  a purely  mechanical  process.  In  part  this  is  true.  \Uien  compre- 
hensive pei’formance  and  error  data  is  available  to  support  the  definition  of  true, 
error  existence  and  detection  or  reduction  probabilities  the  identification  of  effective 
programming  practices  will  be  a highly  structured  activity.  However,  even  under 
these  conditions  it  will  be  necessary  for  a program  manager  to  exercise  professional 
judgement  in  selecting  an  appropriate  mix  of  MPP  to  satisf\  the  particular  software 
development  requirements  with  which  he  is  faced.  'Ihe  selection  procedures  to  be 
defined  depend  upon  practice  effectiveness  evaluation  and  develojiment  environment 
analysis.  The  comparison  process  will  produce  effectiveness  \alues  and  the  program 
manager  will  weigh  these  values  and  the  MI’P  characteristics  (as  discussed  in  Sec- 
tion n of  this  report)  against  the  software  development  environment.  Fsing  the  CSC 
comparison  process  will  provide  program  managers  with  more  objective  and  defini- 
tive data  with  which  to  specify  the  project  MPP  in  preparation  for  software  system 
development. 

Selection  procedures  are  structured  to  promote  definition  of  an  optimum  mix  of  MPP. 
In  the  following  subsections  selection  procalures  are  defined  within  the  context  of 
tiiis  research  project.  Then,  the  Mi’P  to  be  recommended  as  Air  Force  standards 
are  identifieii. 

5.2  SEI.ECTION  I’ROC EDl'UES 

Iliis  discussion  of  selection  procalures  will  focus  on  the  quantitative  aspects  of 
MPP  selection.  Determining  which  i)rogramming  practices  ha\e  maximum  phase 
effectiveness  values  within  a life  cvcle  phase  or  across  the  entire  life  cvele  is  a 
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fairly  straightforward  process.  As  will  be  shown,  consideration  must  be  given  to  the 
effects  of  interaction  among  the  MPP  selected  and  to  their  functional  characteristics. 
On  the  other  hand,  defining  the  process  whereby  a program  manager  assesses  the 
software  development  environment  and  his  resources  in  order  to  identify  the  opti- 
mum set  of  iMPP  to  use  is  another  matter  and  will  not  be  attempted  in  this  report. 
Quantifying  programming  practice  effectiveness  has  given  the  manager  a valuable 
tool  for  the  selection  of  MPP  for  his  project.  However,  it  is  recognized  that  in 
using  this  tool  his  reliance  on  the  phase  effectiveness  values  must  be  tempered  by 
his  experience  and  good  judgement. 

In  preparing  for  MPP  selection  it  is  important  that  the  functional  similarities  of  pro- 
gramming practices  be  considered.  Functional  similarity  refers  to  the  employment 
and  performance  characteristics  of  certain  programming  practices  that  make  them 
functionally  similar.  For  instance,  top  down  design  and  structured  design  are  both 
employed  to  perform  the  same  design  function  and  their  effect  on  the  design  process 
is  essentially  the  same.  IIIPO  diagrams  and  THREADS  also  fit  this  description. 

In  each  case  one  of  these  practices  may  do  the  job  better  than  the  other,  but  in  any 
case  no  one  would  consider  using  both  practices  on  the  same  project.  So,  functional 
similarities  among  the  programming  practices  are  identified  to  aid  in  the  selection 
process.  Figure  5-1  shows  the  programming  practices  defined  in  this  research  pro- 
ject with  the  functionally  similar  subsets  indicated  by  vertical  bars. 

Phase  effectiveness  values  must  be  established  for  all  practices  in  each  applicable 
phase.  As  previously  discussed,  this  research  project  has  led  to  the  definition  of 
phase  effectiveness  values  for  the  Code  and  Checkout  phase.  Selection  of  MPP  for  a 
software  development  project  would  require  phase  effectiveness  values  for  all  life 
cycle  phases.  Effectiveness  values  are  assigned  to  programming  practices  in 
accordance  with  the  phase  applicability  of  the  practices.  Phase  applicability  was 
discussed  in  Section  4 and  shown  in  Figure  4-2.  Selection  of  the  MPP  involves 
choosing  a practice  (or  practices)  in  each  functional  grouping  which  has  the  largest 
sum  of  phase  effectiveness  values  across  the  applicable  phases.  In  cases  where  two 
or  more  practices  within  one  group  have  comparable  effectiveness  it  will  be  up  to 
tlie  program  manager  to  select  the  one  he  prefers  (or  perhaps  use  more  than  one  if 
they  are  compatible).  Also,  a practice  might  be  selected  within  a group  because  it 
is  applicable  in  more  phases  even  though  its  effectiveness  value  sum  is  low.  ITiis 
would  be  the  manager's  selection  perogative.  .\lso,  proper  selection  would  require 
optimization  of  MPP  overall  error  types  within  each  phase. 

5.3  STANDARD  MPP 

A standard  set  of  MI’J’'.s  that  would  be  applicable  to  all  Air  Force  software  engineer- 
ing projects  cannot  be  developed  with  the  current  data  available.  Each  project  must 
be  analyzed  in  view  of  its  characteristics  and  development  environment,  then  a 
proper  set  of  MPPs  could  be  selected  based  on  the  effectiveness  of  those  MPPs 
throughout  the  project  life  cycle.  However,  based  on  the  data  collected  during  this 
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project  the  following  set  of  MPPs  would  be  applicable  to  most  software  development 
projects; 


• Top  Down  Design 

• ITireads 

• Builds 

• Chief  Programmer 

• Programming  Techniques 

• Support  Programs 

• Software  Configuration  Management 

• Program  Reviews 

• Independent  Test  and  Evaluation 

• Project  Reviews 

• Threads  Management  System 
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SEC'nON  6 - RESEARCH  PLAN 


6.1  MPP  RESEARCH  RESUL'rS 

CSC's  software  production  data  research  has  resulted  in  definition  of  a methodology 
for  identification  and  selection  of  a set  of  effective  MPP.  This  research  task  began 
with  an  indepth  review  of  three  CSC  software  development  projects.  Tliese  projects 
were  similar  in  many  respects,  lliey  involved  production  of  software  systems  and 
subsystems  for  command  and  control  application.  'Fliese  were  U.S.  Navy  shipborne 
systems  using  ANA’YK  7 and  20  processors  with  the  computer  programs  written  in 
CMS-2  high  order  language.  Differences  among  the  projects  involved  the  sizes  of 
the  systems  being  developed  and  consequently  the  sizes  of  the  project  staffs.  ITie 
contractual  relationships  betw'een  CSC  and  the  U.S.  Navy  or  the  Navy's  prime  con- 
tractor varied  among  the  projects.  However,  the  differences  most  significant  to 
this  research  involvixl  the  programming  practices  employed  on  each  of  the  three 
projects.  Each  project  staff  used  a different  set  of  practices  either  by  choice  or  by 
direction  of  the  customer. 

Once  these  programming  practices  had  been  identified  and  defined  by  the  re- 

search team,  the  projects  documentation  and  reports  were  reviewed  to  identify  per- 
formance data  supporting  analysis  of  the  practices'  effectiveness.  Tins  information 
included  development  milestone  achievement,  program  test  results,  program  trouble 
reports,  management  review  documents  and  system  size  data.  The  documented  in- 
formation w'as  supplemented  through  interviews  with  project  personnel.  All  of  this 
research  material  was  analyzed  to  isolate  the  data  to  be  used  to  determine  practice 
performance  criteria. 

As  the  research  data  was  reviewed  it  became  more  and  more  apparent  that  research 
of  this  type  cannot  be  conducted  effectively  after  the  fact.  Once  the  software  develop- 
ment plan  has  been  w'ritten  and  development  has  begun  or  been  concluded  it  is  not 
possible  to  produce  complete  and  coherent,  programming  practice  analysis  data. 
'I'hcre  are  several  reasons  for  this,  foremost  of  which  is  the  fact  that  project  mana- 
gers are  almost  totally  concernal  with  producing  and  delivering  the  software.  'ITiey 
assign  and  adjust  resources  to  satisfy  the  most  urgent  requirements  with  tittle  regard 
for  continuity  of  programming  practice  employment.  Also  management  reporting  and 
documentation  is  not  structured  to  support  programming  practice  analysis.  All  of 
this  leads  to  discontinuous  data  for  analysis,  an  unpredictable  research  environment, 
and  marginally  useful  management  reports.  'Iliese  factor.s  prompted  consideration 
and  development  of  a new  approach  to  evaluation  of  programming  practices. 

CSC's  .MPP  eomparison  methodology  is  propcs  ed  as  a comprehensive  and  universal 
approach  to  specificeition  of  proje'ct  programming  practices.  'Hie  prototype  process 
describe’d  in  Section  4 of  this  report  can  reduce  the  identification  and  selection  of 
effective  programming  practices  to  a straightforward,  well  defined  procedure. 
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It  has  been  used  to  produce  the  list  of  standard  MPP  that  is  shown  in  Section  5 of  this 
report,  'lliis  is  still  an  unproved  process,  however.  More  research  and  testing  will 
be  required  to  establish  the  validity  and  reliability  of  the  procedure.  That  research 
and  testing  is  discussed  in  the  following  subsection. 

6.2  FUTURE  UESEAitCH  TASKS 

Perhaps  the  most  notal)le  results  of  this  research  project  is  the  insight  that  has  been 
gained  into  the  definition  of  effective  programming  practices.  As  with  most  learning 
experiences,  this  one  has  not  ended.  'Ibe  research  that  has  been  completed  was  pro- 
ductive, but  there  is  still  more  jinxiuctive  research  that  should  be  done,  'fhe  follow- 
ing discussions  will  outline  the  requirements  for  future  research  into  definition  of 
MPP.  Those  requirements  are: 

• Design  MPP  research  program  in  coordination  with  software 
engineering  project  planning 

• Produce  data  management  system  to  assimilate  and  reduce  research 
data 

• Conduct  analysis  of  the  practices  performance  data  and  recommend 
MPP  for  future  projects. 

6.2.1  Research  Program 

The  MPP  research  program  should  be  desigmxl  to  complement  planned  software 
development  projects  where  the  research  will  take  place.  Research  should  begin 
during  the  planning  phases  for  the  software  development  projects.  The  research  may 
be  conducted  by  U.S.  :\ir  Force  personnel  or  by  a disinterested  contract  research 
group.  In  any  case,  it  should  not  be  conducted  by  the  software  development  contrac- 
tor. Every  effort  should  be  made  to  keep  the  research  unbiased,  'llie  software 
development  projects  should  1h'  varied  in  type  in  order  to  produce  programming  prac- 
tice perforniance  data  from  diffi;ring  environments.  It  is  possible  that  practice 
effectiveness  will,  in  part,  be  a function  of  the  type  of  software  system  being  devel- 
oped. If  so,  this  relations  hi))  should  be  estal)lished . 

The  research  goals  should  be  defined  so  that  the  research  tasks  support  all  data  re- 
quirements. Hopefully  the  r(;search  tasks  and  software  development  tasks  will 
operate  independently  with  minimum  interference.  The  research  must  not  affect  the 
conduct  of  software  development.  Research  data  should  be  collected  from  the  Analy- 
sis through  Installation  Phases.  In  addition,  procedures  should  be  defined  for  con- 
tinuing data  collection  by  tiie  user  into  the  Operating  and  Support  Phase.  This  data 
collection  will  produce  a mass  of  programming  practice  performance  data  for 
reduction  and  analysis.  'Ihe  most  effective  way  to  satisfy  the  requirement  for  reduc- 
tion is  through  the  use  of  a data  management  system. 
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6.2.2  Data  Management  System 

Performance  data  will  be  used  to  determine  the  probabilities  of  error  detection  or 
reduction  and  ultimately  the  practice  effectiveness  values.  The  probabilities  of 
error  existence  must  be  determined  independently  by  the  research  team  during  pro- 
ject research.  All  of  this  data  will  be  cataloged  by  the  data  management  system  and 
manipulated  to  produce  MPP  recommendations  and  reports. 

The  programming  practice  performance  data  will  be  developed  from  program  trouble 
reports.  This  will  be  supplemented  by  other  reports  such  as  those  prepared  for 
milestone  achievement,  unit  test  and  core  budget.  The  program  trouble  reports  and 
other  internal  error  reports  will  be  used  to  determine  the  types  and  volume  of  errors 
being  generated  during  software  development  and  integration.  Further  reporting  and 
analysis  must  be  accomplished  in  order  to  determine  the  relationship  between  the 
occurrence  of  these  errors  and  their  detection  or  reduction  by  the  programming  prac- 
tices. The  other  reports  will  be  used  to  complete  analysis  of  the  practices'  perfor- 
mance. 


6.2.3  Performance  Analysis 


There  are  two  aspects  to  the  analysis  of  programming  practice  performance.  One  is 
the  evaluation  of  the  practices  when  they  are  not  being  used  in  a controlled  environ- 
ment. That  is,  a software  development  project  is  studied  after  it  is  concluded  to  see 
how  well  the  practices  performed.  This  was  done  for  the  Software  Production  Data 
project.  The  second  aspect  involves  specification  of  the  practices  to  be  used  in  a 
development  project  and  monitoring  their  employment. 

Once  the  programming  practices  have  been  assigned  effectiveness  values,  the  validity 
of  these  values  must  be  confirmed.  Their  validity  can  be  confirmed  by  applying  them 
on  a software  development  project  and  analyzing  their  performance  relative  to  error 
detection  or  reduction,  production  milestone  achievement,  program  test  results  and 
project  cost  control.  For  subsequent  projects  the  MPP  can  be  specified  using  the 
results  of  the  MPP  comp^jison  process.  Then  the  performance  analysis  can  be  con- 
ducted as  previously  discussed.  Thus,  through  this  controlled  process  of  experimen- 
tation and  analysis  a viable  and  efficient  MPP  definition  technology  can  be  developed. 
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AUTO-FLOWCHART  PROGRAM 

'ITiis  is  a utility  routine  that  accepts  as  input,  flowchart  statements 
added  to  the  program  code.  Using  these  statements  the  routine 
generates  a standard  flowchart  of  that  program.  Modification  of 
program  logic  which  necessitates  changes  of  program  ctxie  also 
will  affect  the  corresponding  flowchart  statements.  As  the  pro- 
gi'am  code  is  updated  so  also  are  the  flowchart  statements,  thus 
assuring  production  of  current  documentation  through  a dynamic, 
versatile  process. 

BUILD 

A logical  collection  of  functionally  related  processes  that  are  im- 
plemented and  demonstrated  as  a set. 

BASELINE 

A configuration  identification  document  or  set  of  such  documents, 
and  also  the  computer  programs  after  the  product  content  and 
structure  has  been  formally  designated  and  fixed  at  a specific 
time  during  the  product  life  cycle. 

Under  baseline  configuration  management,  there  are  usually  these 
baseline  positions: 

Allocation  - established  at  the  end  of  the  requirements  ■'and  per- 
formance definition  phase. 

Functional  - established  at  the  end  of  the  conceptual  phase, 

normally  existing  prior  to  the  start  of  a software 
development  project. 

Product  - established  at  the  end  of  the  test  and  acceptance  phase. 
CHIEF  PROGRAMMER  TI-iAM 

A management  technique  that  organizes  program  production  staff 
with  a Chief  Programmer  as  a technical  manager  or  'super- 
programmer' responsible  for  the  efforts  of  a program  development 
team. 

CHANGE  .STATUS  REIXIRT  (CSRl 

A listing  or  report  of  all  proposal  changes  and  corrections  to  a 
configuration  aiid  their  current  disposition  and  implementation 
status . 
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DESIGN  TROUBLE  REPORT  (DTR) 

System  desifijn  problems  identified  by  anyone  (CSC  or  Navy) 
associated  with  the  DD-963  project  are  defined  and  documented 
through  DTRs.  DTRs  are  formally  controlled  by  CSC  personnel. 

DESIGN,  TOP  DOWN 

A methodology  for  system  design  whereby  the  high  1 evel , executive 
components  of  the  system  are  designed  first  and  the  balance  of  the 
system  is  designer!  in  an  orderly  progression  from  top  to  bottom. 

DECISION  TABLE 


A table  of  information  consisting  of  all  contingencies  and  actions 
that  are  to  be  considered  for  each  possible  set  of  conditions  in  the 
description  of  a problem.  Decision  tables  may  have  multiple 
dimensions  depending  on  the  complexity  of  the  arguments  and 
actions  inherent  in  the  problem.  Decision  tables  are  sometimes 
used  in  place  of  flowcharts  for  problem  description  and  documen- 
tation. 


DESIGN  REVIEW 


Detailed  Design  Review  normally  occurs  at  the  completion  of  the 
design  of  a system,  subsystem,  or  level  of  abstraction.  Consists 
of  a detail  oc'  examination  of  the  software  design  and  may  be  per- 
formed numerous  times  during  development  of  the  software. 
.Successful  completion  of  this  review  establishes  the  design  base- 
line and  program  coding  begins  at  this  time.  (Also  called  Critical 
Design  Review. ) 

Preliminary  Design  Review  normally  occurs  at  the  completion  of 
the  system  design  phase.  May  be  formal  or  informal  in  structure. 
Successful  completion  of  this  review  establishes  the  preliminary 
of  system  level  computer  program  development  specifications, 
interface  sijccifications  and  data  requirements  specifications  in  the 
system  de.sign  baseline. 

DYNAMIC  CODE  ANALYZER 

Software  routine  that  analyzes  computer  programs  for  errors 
during  execution  of  those  programs. 


A -2 


ENGINEERING  CHANGE  PROPOSAl,  (ECP) 


A formal  request  for  alteration  of  the  confif^u ration  of  a computer 
program  or  other  configuration  item  that  is  delivered,  to  be 
delivered,  or  under  development.  ECPs  are  required  after  for- 
mal establishment  of  the  component  configuration  identification 
(baseline  position). 


INTERFACE 


A common  boundary  between  or  among  system  components. 
Examples  are  a hai'dware  device  which  links  other  system  com- 
ponents or  a specified  storage  area  within  the  processor  which  is 
accessed  by  two  or  more  computer  programs. 

INTEGRATION  TEST 

A test  to  determine  the  functional  compatibility  of  two  or  more 
dissimilar  systems  or  system  elements  (i.e.  , hardware  and 
software,  applications  and  system  software,  etc.).  Also  used  to 
ensure  the  compatibility  and  proper  functioning  of  a new  or  modi- 
fied system  element  when  it  is  placed  within  the  existing  system. 


MR.  ESTONE 


A well-defined  interim  or  final  goal  which  must  be  achieved  during 
the  development  of  a hardware  or  software  system.  It  usually 
establishes  a baseline  position  or  signifies  the  comnletion  of  a 
specific  task.  Completion  of  system  Builds  is  one  form  of  mile- 
stone achievement. 

MU. ESTONE  REPORTING 

Milestone  Reporting  encompasses  all  levels  of  the  managerial  and 
technical  staff.  At  the  lowest  I evel , the  programmer  milestones 
are  established.  As  the  chain  of  command  progresses,  milestones 
become  broader  and  more  complex,  and  they  reflect  the  status  of 
the  entire  project. 

PROCESSING,  COMPUTER  PROGRAM 
Batch  Processing 

(1)  Pertaining  to  the  technique  of  executing  a set  of  computer 
programs  such  that  each  is  completal  before  the  next 
program  of  the  set  is  started. 
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(2)  Pei'taining  to  the  sequential  input  of  computer  programs  or 
data. 

(3)  Loosely,  the  execution  of  computer  progi’ams  serially. 
Stacked  Job  Processing 

A technique  that  permits  multiple  job  definitions  to  be  grouped 
(stacked)  for  presentation  to  the  system,  which  automatically 
recognizes  the  jobs,  one  after  the  other.  More  advanced  systems 
allow  job  definition  to  be  added  to  the  group  (stack)  at  any  time  and 
from  any  source,  while  honoring  priorities. 


PROGRAMMING 


Defensive  Programming,  is  a concept  closely  related  to  functional 
and  structural  progi’amming,  in  that  emphasis  is  placed  on  avoid- 
ance of  errors  at  the  time  of  coding,  rather  than  during  program 
checkout.  ITie  idea  is  to  visualize  worst  case  situations  and  pro- 
gram accordingly. 

Functional  Progx’amming  emphasizes  the  creation  of  correct  pro- 
grams so  that  much  of  the  program  testing  traditionallv'  associated 
with  building  large  systems  will  not  be  necessary.  Tlie  methodol- 
ogy focuses  on  clearly  describing  an  object  in  functional  terms  be- 
fore it  exists  in  final  form. 

Peer  Group  Programming  is  a concept  usexi  by  CSC  which  focuses 
on  the  fact  that  it  is  often  helpful  and  instructive  for  a programmer 
to  have  his  program  rcviewal  by  a group  of  peers,  both  before  and 
after  coding  and  prior  to  checkout. 

Structural  Th’ogramming  is  a programming  discipline  providing  a 
means  of  expressing  a system  de.sign  that  ensures  a testable  and 
understandable  implementation,  and  that  enforces  simple  and 
well -definal  connections  between  program  modules.  Tlie  pro- 
gramming discipline  uses  the  repeatal  application  of  a small  num- 
ber of  basic  control  statements  to  form  simple  program  constructs 
that  represent  large  and  complex  programs.  Tlie  process  of 
creating  the  program  modules  includes:  making  local  or  tactical 
programming  decisions  within  thedesignai  module:  writing  pro- 
gram segments  that  represent  tliese  decisions : and  integrating 
program  segments  into  a unit  corresponding  to  a system  module. 

Modular  Ih’Ograinming  is  a division  of  software  into  blocks  of  code 
callal  modules.  Each  module  represents  a specific  subsystem  and 
is  develop(.>:i  a.s  an  entity. 
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"Top-Down"  ProKramniin^i;  is  a technique  of  software  development 
that  closely  parallels  the  natural  approach  conventionally  used  for 
system  design.  The  control  architecture,  including  the  executive 
program,  major  executive  interfaces  (modules),  and  data  base  is 
put  into  place  first,  llie  internal  control  logic  is  then  constructed 
within  each  module  to  a progressively  lower  level  of  functional 
detail,  ultimately  resulting  in  a completely  coded  software  package. 

PROJECT  MANAGEMENT  PLAN  (PMP) 

Project  Management  Plan  includes  compl ete  development  con- 
siderations which  encompass  the  plans  for  phasing,  organizing, 
testing,  change  control,  documenting,  training,  and  installing. 

The  plan  becomes  the  road  map  and  is  the  key  tool  used  for  the 
successful  implementation  of  all  software  development  projects. 

RESOLUCE  UTILIZATION  (PLAN) 

Referring  to  a plan  for  the  assignment  of  resources  (men,  machine, 
money,  time,  and  materials)  to  ))rojuct  elements  (task,  products, 
accounts  and  organization  units),  and  the  accounting  for  the  actual 
expenditures  of  these  resources  in  the  accomplishment  of  the 
project  mission. 

SOFTWARE  TROUBLE  REPORT  (S'lR) 

A form  used  to  define  and  document  software  failures  and  faults. 
TOEDENT  project  called  the  form  itself  an  STR,  on  DD-9ti3,  it 
was  known  as  CSC  lYouble  Report  (CTO).  In  both  cases  it  served 
the  same  purpose.  Tlirough  this  form  project  managers  are  able 
to  track  the  actions  taken  to  dispose  of  software  problems.  It 
also  gives  some  measure  of  programming  effectiveness.  Trouble 
reports  provide  visibility  to  the  customer  concerning  development 
activity. 

SYSTEM  VALIDA'nON  DIAGRAM  (SVD) 


A block  diagram  portraying  finite  jU'ocessos  at  their  lowest  s\iL. 
divisions.  Each  block  (stimulus-response  el ementt  depicts  in;  t 
(stimulus),  process  (operation)  and  output  (response).  A ; 
pi  ete  function  may  be  represented  by  a series  of  blocks 
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SYSTEM 


(1)  An  assemblj^  of  methods,  procedures,  programs,  or  tech- 
niques united  by  regulated  interaction  to  form  an  organized 
whole. 

(2)  An  organized  collection  of  men,  machines,  and  methods  re- 
quired to  accomplish  a set  of  specific  functions. 

(3)  A structured  combination  of  interacting  parts  satisfying  a 
set  of  functional  objectives  and  performance  objectives. 


STUB 


A minimal  piece  of  code  used  to  simulate  an  operation.  In  top- 
down  program  development,  stubs  are  the  first  realization  of  a 
program's  functional  specification.  They  are  expanded  into  pro- 
grams when  the  development  effort  moves  to  the  level  of  abstrac- 
tion to  which  the  stubs  belong. 

SOFTWARE  FAULT 

A departure  from  correct,  specified  program  execution  which  is 
due  to  error(s)  occurring  during  the  translation  of  the  original 
specification  of  an  algorithm  to  the  program  being  executed. 

SOFTWARE  ERROR 

Any  discrepancy  between  a computed,  observed  or  measured  quan- 
tity and  its  true,  specified,  or  theoretically  correct  value.  Errors 
are  introduced  into  software  by  human  mistakes,  that  is;  (1)  de- 
ficiencies or  misinterpretations  of  design  criteria , (2)  logical 
mistakes,  (3)  a syntatical  mistake  and  (4)  mistakes  made  in  trans- 
cribing program  statements  into  the  input  data.  Contrast  with 
fault,  malfunction,  and  mistake. 

SOFTWARE  DEVELOPMENT  LIFE  CYCLE 

The  process  by  which  user  requirements  are  translated,  via 
software,  into  a functioning  system.  Tlie  actual  steps  involved 
may  differ  according  to  the  size,  purpose,  and  end  use  of  the 
software.  For  the  formal  development  of  a large  program  the 
following  steps  are  involved;  system  requirements  definition, 
software  requirements  definition,  preliminary  design,  analysis 
and  detailed  design,  coding  and  checkout,  development  and  system 
testing,  delivery,  and  maintenance. 
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SOFTWARE 


Computer  programs,  procedures,  data,  and  associated  documenta- 
tion concerned  with  the  operation  of  a data  processing  system.  The 
term  software  includes  (a)  computer  program  products  in  the  form 
of  card  decks  and  magnetic  tapes,  and  (b)  all  documentation  asso- 
ciated with  computer  programs,  including  specifications,  listings, 
manuals,  flowcharts,  version  description  documents,  test  plans, 
and  test  procedures. 


SIMULATOR 

A software/hardware  system  whose  primary  function  is  to  emulate 
the  behavior  or  operation  of  another  system.  Simulators  are  nor- 
mally employed  to  provide  realistic  inputs  for  a target  system  or 
device  which  is  being  evaluated.  They  may  also  be  used  in  pure 
research  to  mimic  other  systems’  responses  to  programmed  in- 
puts when  the  real  systems  cannot  be  made  to  respond  under 
laboratory  conditions. 

SEGMENT 


Portion  of  a larger  system  entity,  such  as  a subsystem,  program, 
or  data  segment.  Segment  dimensions  may  be  somewhat  arbitrary 
since  a segment  usually  encompasses  a single  functional  operation. 
Tbe  term  segment  is  most  often  applied  to  individual  subroutines 
or  subelements  of  system  modules. 

SOFTWARE  CONFIGUR.^TION  MANAGEMENT  (PLAN)  [SCMP] 

Software  Configuration  Management  (SCM),  is  a tool  that  assures 
the  integrity  of  implemented  systems.  SCM  locks  out  the  pro- 
grammer from  the  implemented  data  files  and  programs  by 
establishing  controlled  program  versions,  and  it  allows  only 
authorized  changes  to  those  versions. 

STRUCTL’RED  PROGRAMMING 


(See  Programming) 


STIIUCILRAL  AN.M.Y7.ER 

Software  routine  that  analyzes  structured  code  to  assure  that  the 
basic  structured  patterns  are  adhered  to. 
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TEST  SCRIPT 
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A procedural  scenario  using  Threads  to  ensure  all  requirements 
of  the  program  specifications  are  met. 
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TOP-DOWN  DEVELOPMENT 

The  process  of  designing  a software  system  through  a sequence  of 
step-wise  refinement  rhough  successive  levels  of  abstraction;  and 
then  implementing  the  design  by  giving  development  priority  to 
control  structures  and  module  interfaces.  Application  routines 
(i.e. , bottom-level  modules)  are  first  represented  by  program 
stubs  simulating  the  control  and  interaction  interfaces.  The 
actual  programs  are  subsequently  integrated  into  the  control  struc- 
ture such  that  system  level  testing  for  the  control  structure  and 
interfaces  are  performed  first  and  continually  verified  as  applica- 
tion routines  are  substituted  for  their  stub  representations. 

TEST  CASE  GENERATORS 

Test  Case  Generators  are  utility  routines  that  generate  sample 
input  data  as  pail:  of  the  checkout  activity.  Emphasis  is  on  pro- 
ducing data  which  will  cover  all  conditional  paths. 


THREADS 


The  THRE/VDS  methodology'  is  based  on  defining  a system  in  terms 
of  processes  specifically  keyed  to  required  system  capabilities  and 
then  organizing  all  system  development  activities  around  the  pro- 
cesses. A Thread  is  a functional  path  or  sequence  of  activities 
that  results  in  the  production  of  a required  response  from  a set  of 
inputs  or  stimuli.  Each  requirement  places  on  the  system  can  be 
expressed  as  a 'fhread. 

VERIFICATION,  BOTTOM-UP 

This  process  was  used  on  the  DD-963  project.  It  is  accomplished 
by  first  verifying  that  the  program  auto  flowcharter  statements 
match  the  program  logic.  The  auto  flowchart  routine  will  then  be 
executed  to  generate  program  flowcharts,  "niese  flowcharts  are 
then  compared  to  the  design  specification  to  ensure  conformance. 
This  process  is  usually  accomplished  by  someone  other  than  the 
responsible  programmer. 
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VERIFICATION  & VALIDATION  (V&V) 

Verification 

(1)  The  process  of  testing  designed  software  to  ensure  that  the 
system  and/or  system  components  are  logically  correct. 

(c.)  Hie  process  by  which  the  contents  of  a computer  program's 
physical  medium  (tape,  deck,  etc.)  are  authenticated. 

Validation 

The  process  of  ensuring  that  specific  program  functions  meet 
requirement  specifications. 
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